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chapter t 

"■ INTRODUCTION ; 

The Need for an Integrated H'^^qginQnt System 

The federal-state employment security system of the United States, 
consisting pf the U. S, Employment Service (ES) , the Unemployment 
Insurance Service (UIS) , and the 54 'state .and territorial' employ- 
ment security agencies, recently celebrated its 40th birthday. 
During its history, the system has assisted the nation in coping^ 
with the problems of the Great Depj^ession and other less severe - ^ 
economic retressions, has supported the recruitment an(5 relocation 
of manpower during wars and has played a significant role in the ■ 
conduct of manpower development and training programs. : yhrough-- 
out this history, while the system has responded to changes^ in 
natiojial heeds and national priorities, thfe ES has confTlnued to * 
perfprm basic labor functions and the UIS has continued. to adminis- 
ter tfte Ufiemployment compeiisation system, both in accordance with ^ 
the original mandates of fhe Wagnef-Peyser Act, 

In recent years, the government lias developed a number of alternate 
structured for performing loany of th^ ancillary functions wjtiich - 
accrued to the* &S and UIS over the years., The employment security 
system has been directed to focus its attention *more directly on" 
the services to employers, un^ployed persons, and job sjeeker^ 
than at any time since I'SeOt Even within the somewhat narrower 
scope of activity, the system expends approximately $1 billion 
per^ year in administrative expenses, exclusive of iinemployment 
contlpensation costs. . * * 

' At the same tiineV the* administratidn has re^quested thai improve^d 
management^ and funding allocation methods be developed and applied. 
The UIS has developed a qost model, based (implicitly) On the 
concept that the UIS is a factory with measurable and concrete - 
in^puts (employer contributions ^nd claims) , concrete output's 
(monetary and nonmonetary determinations, chj&cks, and employer 
tax statements) , and clearly definable production processes which 
can be optimizedt'^ The cost model is used to allocate UI funds 
based on expected workloads and staffing requirements derived 
from estimated minutes-perrunit, taking into account differing 
legal and administrative requirements in the different states. 
The cost model has been accepted as fair and reasonable by the 
states and by 0MB for funding allocations. ' . 

While the ES has developed a performance-based allocation method and* 
a number of m^glnagement to61s, the ES ddes hot yet have a compre- 
hensive mana^emervt and budgeting system which is accepted by 0MB 
and the states* A major reason is, of course, that the ES problem 
is inherently more complex. The ES has undertaken two ^iii'tiatives , 
both currently incorporated as Presidential objectives, to iinprove 
funding allocation methods and management* processes * 



The firsit of these is the Balanced Placement Formula (BPF) . First 
introduced in a limited way during the latter part of FY 19T3, 
the BPF was applied nationally i^ the allocation of ES grants for 
FY 1975 and, with research^based revisions, for FY I'^Ve. **The 
formula is^ designed to measune performance of each State agency 
during th^ most recent period for which data are available and to 
allocate funds for the operation of the agency,, based On this 
perfQormance." ( BPF FY 1976 Handbook , p.l) 

The seconci is a research and devielopment program to develop per- 
formance st^dards for the ES*^ This volume reports the methods 
and techniques developed in the first phase of that program. As 
stated in the original Request for proposals: 

The priraary objective of the demonstration project is to^ develop 
valid performance standards for Job Placement activit;ies and the 
Placement Support Service functions* These standards will be 
used to reflect the effectiveness and the efficiency levels at 
which the' various placement and support functions of the ES should 
be performed with consideration to bat^i ^antity and quality; ^o 
design a feasible system of methods and procedures for usin^ 8uch 
measures to assess the performance of state agencies in performing 
activities relative to the standards; and to design a system for 
the periodic revalidation of the standards* 

^^fter: the project was started^ BPF research was added as dn 
explicit component* The results of the combined. researxih fotm 
the basis for the present report* 

The best s^ummary of the current status of the BPF and the per- 
formance" standards development is contained in the BPF FV 1976 
Handbook / in a discussion of future directions for improvement, 
p^ts of which are extracted , here (pp*- 8-10): 

* 

0 The formula may well be based in part on past performance and on 
potential fqrxserving clients during the budget. year. Research 
is expected to be carried out on ways to measure such potential 
validly, based on available resources* A major'need %z data 
new hires in a-ll industries instead of merely manufacturing* The 
. allocations would be contingent in part on experience of the state' 
in atj:aining the projected potential in each quarter of the 
budget year* 

* ^ o Performance of each State may well be measured ag^nsth standards ^ 

for the State, instead of ag^i^st national, exper/ence l(or average) . ^ 
The performance standard^^ project now underway amd one planned for 
the immediate future are expected to provide* thfe methodology and 
the d!^ta needed for output petformajic ^^^€uid^g gs^* J 

o Consideration may be given to grbuping^' States according to simi- 
larity of size ^nd homogenei^ty of environmental factors, as an 
interim replacement for national average* This was attempted 
for FY 1976 but suitable guidelines for valid groupings of . 
States are '^ot yet avail^le* ' ^ ^ 

6 ' 



• Extern^^l factors, affecting productivi^ty of State agencies (insttead 
pf or in addition to unemployment) are likely to be built into 
future fomulasi based on currently ongoing and projected research. 
Factors being considered include character i a tic? of job seekers, 
industrial mix^ occupational mlxj growth- in employmenti labbr . market 
^i' ^ conditions; size of I^or force, and the like. 

5 " 

« Guideline^ will likely be provide^ for applying the Balanced 

Placement Formula to labor market area's and to local, of fices on a 
quarterly basi^ to detect strengths and shortcomings and to 
indicate corrective actions needed. 

State, regional and national ES administrators and managers have 
generally accepted the concepts of performance standards and of a , 
performance-based budgeting systetn, and they have supported 
improvements of the type described above, intended J:o correct foV 
perceived inequities and inadequacies of the current bPF. To fully 
realise and utilize their potential, standards must be developed 
and applied in the context of ^n integrated management system 
concept, whifeh relates output standards not only to budgeting but 
also to performance evaluation, the diagnosis of the causes of poor 
performance, and the development of performance improvement plans. 
Many of the needed elements of an integrated management System ' 
exist or are under development - output standards, input standards, 
m^fthods for analyzing local office location and management practices, 
and the existing self-appraisal process* 

The gurpose of this volume is to set forth a conceptual framework for 
an integrated management system, to, describe the eiemen^st of such 
a system that can be efficiently developed in the short^^un, and 
to present methodologies for developing and using eleiTijfints not 
documented elsewhere* 

0 Chapter II presents a conceptual framewprk for the 
development process. 

o Chapter III discusses alternate methods for developijig 
models for projecting ES output (placement) performance^ 
adjusted for the influence of external economic a,nd 
other factors; and refining performance-based fund 
allocation system (BPF) • Methods are presented for 
using the results of such mod^els to: 

a* Con^te expected ES performance at the labor area 
level and to set output standards. 

b. Simulate ES system response to changes in economic 
conditions and budget levels. 

• c. -From differences between actual and standard 

porfomance at labor area level, compute weighted 
, sum to obtain a staJre summary score of actual 

performance compared to standard^for BPF allocation. 




d. As a byproduct, . C^lculat^ and print funding alloca- * 
^ ^ , tionsj^ including performance-adjusted supplemental 

* > alrl^cationii . 

© Chapter IV provides detailed methods and plana* which can^^ 
be used to develop input resour<qjfe -utilization guidelines. 
Specific elements aret 

, . . * / 

a. Sampling to develop input models^based on resource 
utilization patterns of areas whose Qutj)ut performance 
is high compared with standards adjusted for external 
factors. * V ^ 

b. Measuring staf j^^ tijge utilization, service percentages 
and key quality factors. 

c. Analysis of data to develop initial input standards 
for different types, of labor areas. ^ ■ ^' 

d. A<rgregating and disaggregating input standards for 
local, State and national use. 

« Chapter V returns to the theme of the integrated manage-- 
^' roent system and discusses the* need to refine the system 

* ^or performance evali;ation and improvement. It contains 
the following topics: ' * . 

a* A iSchedule for developijig initial output and input 
* standards during, the next year. 

b. The neecj for validation and. revalidation of performance 
standards. | * ' 

c. Limitations on performance- standards and future ^ 
development needs i ^ ^ . ' " 

' & ■ . * L 

It should be noted at the outset and borne in mind throughout 
that, the prijnary focus of Chapters III, IV, and V is updn 
specific steps that are believed feasible- fo^: implementation in ■ ^ 
the short run (i.e., within a one-year time ^rame) , to dmprove 
ES management systems. The i?deal long range approaches /and the 
limitations of recommended short run steps are presented in < 
Chapter II. The steps recommended in the remainder' of the volume 
are designed to permit the. development and implementation of new 
approaches by the beginning of FY 1977, given the current state , 
of knowj-edge, while at the sartie time beginning a major evolutionary 
* ^ process that may taJce several years to complete. 
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x/^ ■ , chapter ii ' . - 

■ \ - ' \ ^ 

CONCEPTUAL FRAMEWORK ' 

The U. St Employment Service is an organization .whose overall 
miss^ion may described as providing labor market services and 
information to individuals, employers and other organizations* 
As noted in the Introduction, the specific purposes and objec'-^ 
tives of the ES within this general mission have changed from^ 
time to tif^^ an^ may be expected to change in the future in 
response to national needs and priorities. Yet at any, given 
point in time, the ES consists of a bundle of ^organizational 
and staff resoutqes that must be managed to meet current object . 
tive^. The problem is to define the elements of an integrated 
management system such that the system itself can remain relatively 
stable and provide tools fojc management , not only^ in response to 
a specific set of objectives but also in response to changes in 
objectives* ' ^ 

'I 

For present purposes, the bas^c framework chosen is tha^ of the 
ES as a labor efxchange whose basic functions, processes and 
classes of outcomes, are assumed to be relatively constant over 
time* Th4 discussion of the ES labor exchange functions is baseci ■ 
upon two assumptions wtuch are fundamental *to the development' of 
the^ integrated managem^t system mdaels and methods presented 
subsequently in this volume * ' ^ 

First, 4he ES is a federal, state and local system , with each 
level cyonstitutin^ a subsystem having appropriate rol^s and 
functions »: Second, the labor exchange (and related)^ functions 
exist only a^t the labor area Tevel t " "definition, a labor area or 
SMSA is an iiitegrated social and economic system constituting 
the relevant economic and sodlal environment within which ES 
labor exchange functions are actually performed and ES services 
delivered.* # ^ ^ 

Therefore, the federal and state levels of the ES system exist ^ 
only to perform support junctions (funding, interpretation of 
pOl^^ry and law, training ahd technical aSsistancte, management 
v.and evaluation) whose effect, however crucial,^ can only be 
^leasured in a9tual performance at the labor areaJlevel* 

Within this context,'^ the definition of specific purposes and 
objectives implies a series of related research and analysis 
steps that would be required to define specific configurations' 
of ES resources optimally organized at the labor area level to 
meet those objectives. As •;an interim st^, it is possible to 
de^eldp tools for measuring how efficiently ES resources are 
being utilized to meet the current constellatiorf of (loosely 
defined) objectives. The chapter discusses each of these 
subjects in turn. 

^ As used in this re^rt, the term "labor area" is synonomous with the term 
"SMSA" (Standard Metropolitan Statistical Area) f which is a defined labor area 
ooAs^isting of a central city with at least 50,000 population and surrounding 
counties. Much 'of the data needed is not regularly dbmpiled for 'smaller areas. 
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A, 0^^erv^ew of ES Labor Exchange Functions - * . ^ - 

A convenient model of th'e ES at the operational, level is that 
of a production process. Figure 1 on the following page ' * , 
provides an. overview of the ES labor exchange, function, qhpwing 
generalized inputs, processes Vnd outputs, 1 ^ * ' "^^ 

l,^ External Environment 

'A - * 

The ex^rnal environment of the ES labor exchange at the A 
, labor e^rea lavel ' includes all the economic ajid .socifal ' , » 
conditions over whiph the ES has little or n© control: 
growth, unemployment, industry mix and labor force 
"composition, alternate placement channelSt As indicated 
' oa the diagram by %\ib ""^r^rows, thfe external anvironn^ental 

* factors ard hypothesized to impact on every 'jaspeqji^ 

* activity, from Bfetermining the size and ps^mj^^S^ 
of tt^e popls of job seekers and job open^h^gs ^3^<3|Jft:^' ' 
the ES (draws its us^rs^'to affecting 1:]ie success ofT^ 
ps in producing outcomes, * Z 

Internal Environmerit. " . ^ ' . 




ES performance at thja labor areV level- may be /constrained 
by factors that are/ internal to the federal-state systefm as . 
a whole f but which are not under the C0ntro]|^ of local ES 
mai^gement: policies and laws, history, management 
structures^ human and physical resources^ The elements' 
of thte internal environment affect tWe identification ^ 
processes (and therefd^e the compositiqn of available 
users), thei service techniques, and ultiltiately the quantity 
and quality of ES outputs, ^ 

3. |-^^*9tential^ Users 6f ES Services and Identification Processes 

In, the absence of clearly defined objectives, the potentisrl 
users of the ES include all jgb seekers and all available 
job openings in a given labor area. A variety of identifi- 
cation processes results in the registration of appliccha!^s. ^ 
and the listing of job orders, from the potential pools. 
The BS does not necessarily atciampt .to register^ all jdb 
seejcers nor to list all job openings* The specific subset 
sought and.obtained and the processes used vary with ch^tnges 
in ES objectives, laws and policies, and tescS^rces. 

■4t Production Processes ^ . > r t 
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the ES attempts to match available applicants and job 
openings through a variety of techniques which can be 
called production processes. The categorization in 
Chapter XV of this voluir-e identifies 13 discrete, 
measurable placement and placement support, functions 
(which include identification- processes) These processes 
both utilize existing state-of^t^he^art and labor market 
information as resources to the placement process and 
(potentially) contribut^^to the development of thesfe 

€ . sj 

.10 " 



Figure I 1-1 



Blocfe Diagram ' . \ ' 
Emplo'i^Tnent S^^rvice Laopr Exchange Functions 



t ^ ^ ' . ' EXTERNAL fitfVIROMtaiNT ^ ^ . 

Uneinplo^gnen£| Gyowth^ Industry and- LabQi^Force.CQinpqsl.tioni Al^m^te placei^i^ Qhannels^^ etc> \ 




resoiJrceaV^ Therefore, improved techniques "and better^ 
laboxTnarH^it information are (potential) direct byproducts^, 
of the basic labor exchange function, whether: of not tUese 
outputs meet specific ES objectives for internal ii^iprove- 
ment or ^rvice to other organizations, 

5 t putpui _ . . ^ 

The outpttts from the system can^be classified into two 
categories for practical areasons: 



a. Applicants placed hnd job openings filled 

' ■ ■ * " J/ 

These two types of outputs, includinq^ si4)-ciasfe%s ^ f 
of each, are the only ES outputs which atp currently 
Ineasnred oA a regular basis. As two sides of a coin, 
both me'asures in"dic%te^the CQmpletioh cTf a man-'job 
matching transaption and therefore indicate positive^ 
concrete results from the. j^S prgduction *pj:ocess , even 
■ though Only limited data exist ^ the^ qul^ity "and 
permanen<je^ of each match ,^ 

b'. Other *short-term and long-term outputs •re.lated to 
_ app*licants:aind employers ' * ^ : 

Short-term outp.uts: iifiproved vocational /prepar^tionj 
or vocational choice, bettex^'job search ""techniques , . 
bettet labor market information, better job descrip- 
tions, etc, 

Long^T-tertn" outputs :* increased lifetimes darnings or 
incbmes, reduced' unemployment*, higher productivity^, 
increased family stability, et^p*- . - ^ * r 

; ^xistiilig^datA Ofc>llection- and reporting systems lacR^ 
„thfe (r^lativef expen^ivfe) folloi^up data /and oont-ror 
\ grouj) data 'ijec^eSafiiy. to measure^ the non^^placement 
output^ .and -^^^testj-mate' lon^-rut?*'ef f e^ctivenes^ of ^ 
placeipent-i^elatjf/i ou^ffatsv , ' > ^ 



To be truly coinpi^fehensi^e and .complete , an^ifttegrated management 
systesn should be bas^d upon a hiodel o£ the^ES system that 
completely specifies all o&j^ctives; quantifies the potential ^ 
for ES Services in ^ach labor area pUrsuan^t to tfeose objectives; 
defines the optimum ^configuration of* riespujrces to meet that 
potential, and measures aid ES outputgy ^ ^ _ { 



In concept , " such a model could^ be. developed from a series of . ' 
hypotheses relating to each and every, caijsal linkage indicate?? ' 
by an- arrow in Figure^ LI-1, and would con&ist of a series of ^ 
structural equations describing the ..hypothesizeja r^a:tiori5liips. 
Figure II-2 lists sbme ' df the tiypothesefe tl\at shout^^rovide . 
the btaS^s /or constructing and testing a model thaft relates 

:-\; 12 ■ ■. 
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Figure 11-2 

H^nDOthe3e8 Relating to 
Environmental FactOjg s 
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envijTonmental factors to ES output performance. tn al} candor, 
we must admit*that t^e table of hypotheses was Constructed 
after completion* of the statistical analysis, not before. 
Therefore , some elejnen^s of thei tested model are a^ vari^ance 
wLth the hypotheses, and the^ sta(€istical results presented in 
this report are Consequently deficient. Even 'so, the liitiit^d 
results presented and the hypotheses*^ stated provide the basisy 
for future development of t^he model.. , ' ^ 

In preparing this'report, it was not possible to fully -"Elaborate 
the structural Tiiodel and a detailed plan for developing it<b 
Therefore Section B merely outlines some pf the research that 
woul<f be necessary to develop such a model/ for jthe.putpose of 
identifying some of the practical limitations^ of pur* recommended 
short-run approach imposed by constraints of;da£a, 1:imey and 
resoiCrces. ^ * V 

An Ideal Development Program \ 

Ideally, there are three hier^rc^ic^td' levels of research whiQh 
could and should be performedt The highest level question -is, 
shoulj3 tjiere be a*public agency involved in the labor exchange 
function, and if so, how shoul4 it b^st ruptured- Calong the 
lines of, the existing public employment service or along other 
lines)? At: the next level, a researchable question is, given 
thAt we have an ES , is* it operating in a cost-ef f ectiv^n^anner 
to/ realize agreed-upon objectives? At the bottom^ level ,'Ngiven 
tnat we have an ES but only limited indicators of its effec-V 
tlvehess, are there any tool^s which we can use to better pro^ct 
performance , allocate resoul'ces, identify areas of. poor perfor- 
mance , 'diagnose the. causes and develop performance improvement 
plans?" ^ , 

If one were to assume the need and start from scratch to ^ * 
est^lish a public agency to perform labbr exchange functions, 
^or 'to re^^define and redirect the ES , one would follow the steps, 
implied by the diagram on the following page. ^ 

It First, one would define or clarify the missions, pur|>Qses, 
an& objectives of the agencyt- The objectives might include 
correction of inequities in the lafcor market , improved 
^quality and availability of labar market information, 
reduction in -unemplo^taent, etc., all of which have been 
objectives of the ES at different times with differing, 
degrees of emphasiSt The definition of objectives would 
: be used to define specific target populations for service, 
service modalities*^, agency functions^ 
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Figure. 1 1-3 



Block Diagram of an Ideal 
' Development Approach 
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2 s Thp second step would be to qcgiduct a thorough and ■ 
detailed analysis of the^ potential market for each 
function and servicfe of the agency in each labor \ 
market. The. analysis would take into account the" 
existence of alternate ^ufaiilp and private agencies 
. fulfilling one or more of tho objectives, and the / 
* probability .and desirability 'of competing success- 
fully vith sVch existing agencies. The ^results bf 
' the analyjiis would indicate what geographic markets 
the public ageh^ (ES) should .attejnpt to serve, what 
■'. Segments of these marlcets should be pursued and what 
the maximum potential wodld bfe^ for achieving each 
objective in each^ market . Resources could then be ■ 
allocated (or reallocated) and specific techrfiques 
could be developed (or refined") to achieve maximum 
performance in relation to the expected potential/ 

3; At the same time, a management system wou^d be developed 
(or refined) to permit measurement of actual performance 
in relation to potential, to assess the efficiency with" 
which resources were utilized, to ^J^pa're regular evalua- 
tions of the cost-effectiveness of the agency in relation 
to jfeach objective and markelj, and-^o provide the basis 
for reallocating resources. This system would include 
several types of evaluations (polic^, program and opera- 
tions) . As" the system developed, ^|>ecific standards 
would be established for output performance and for input 
resource utilisation configurations in support of each 
output'. ' 

Imfflementation of such a program would require significant 
amounts of time and resources. Precise definitions of ES 
purposes and objectives do nob axist, and based on recent 
indications* such definitions will be a long time coming. 
Little market analysis has - been attempted to define ES 
potential in labor areas, in part because relevant data on 
labor force and employment does not exist. ES service 
techniques have often b^en studied and "evaluated", but 
never in a comprehensive labor exchange model that could 
permit estimation of the pai:tial contribution of each , service 
to measured outputs. ^ And rib ongoing system exists for ^ 
measuring, long-term ES outputs, a necessary component for 
an evaluation system. 
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Curtis C» Alleri et al , An Employment Service Research and Development 
Strategy f (Center for Applied Manpower Research, March 1975) esp. Chapters 
III and IV; ^md the presentations and discussions at the Joint Conference 
of the Manpower Administration and the Interstate Conference of Employment 
Security Agencies on *'The Role of the Public Hioployment Service 1975*1985", 
Chicago, April 22*24, 1975. - | g 
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In actual practice, it may require 3 to 5 years to develop, 
a truly comprehensive^ integrated management system with all 
components rigorously specified, thorouahly researched and 
tested. In the meantime, the ES is consuirting resources 
at the rat^ of about $500 million per year, is experiencing 
' reductions in real anaual resource allocations, and Is under 
intense pressure frorq 0MB to improve its operations. It is 
clear that the ES should" not wait to beg^^n the development; of 
snich a systejn, nor should it delay implementation, of such 
elements of a system as may be developed and validated in the 
interim. Thie next section of this chapter outlines recommended 
interim Steps . " ' * 

cl An^ Approacli 

fr\To meet immediate needs of ES executives for improved methods 
V of budgeting, pr03ecting' performance, and developing per- 
fotmance improvement program^, a phased development program 
should be undertaken. Program objectives should be defined 
at each stage of the process to ^oduce useful results 
immediately, while setting the stag^ fo£ refinement ^nd 
improvement iin thje ^next, phase. The initial phase would: 
ujti^ize Methodologies developed to date to meet needs in the 
s^rt-run, while, at the same time developing o^ refining 
methodologies ^or a next pljase. 
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A Constellation of ^^^sting ^hort-run needs which "must be met 
within the next fieral year revolve around the establishment 
and utilization of standards for ES outj^ut performance and for 
input resource utilization.* It would be /desirable to set such 
standards taJcing into account reliable estimates of ;ES patent^-aj 
and the results of th^e evaluation of the ability of the ES to 
realize that potential, but such data will not available. 
The diagram on the following page indicates' specif ic develop- 
ment^ steps which appear to be feasible to undertake," complete 
and consolidate into an integrated management jsystem during the 
next 18 months. 

We recommend. €hat the process begin by setting output standards 
or norms based on average ES performance as measured by 
indicators ot ES ef feffctiveftess derived from extant measures 
of ES performance. This approach has already been used ip ^the 
funding allocation process. for FY75 and FY76, where the BPF 
measured the performance of each state as compared wij:h the ^ 
national average. For the next s^ep we recommend computing 
adjusted performance averages- at the labor area level. Exte^rnal 
economic and policy factors would be used to adjust the average 
for each labor area and to calculate expected performance levels 
according to the methodology described in Chapter ni. The 
adjusted performance averages would be the basis for output 
standards or expected performance levels. 
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ES Integr^teW Management System 
Overview of the Initial Development and Util'i^ation Proceps 
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The standards could be used /or several purposes with actual 
uses at any point in timet depending upon Stage of development 
and level of acceptance:. 
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1. Differences between act^ial and standard performance 

• identify specific labor areas exhibiting substandard 
* performance and indicate . where other diagnostic; 

tools ^hould be, used to identify the causes of poor . • 
performance, etg- , input standards, 

.2t Tbe^ di^ferenceft provide the basis for developing 
input resource utilization j^tandards (minutes per 

* unit, service r^atios and key quality factors) by 
permitting the grouping of 4^eas into Jiigh, medium 
and low performance categ9ries (adjusted for e^cternai 
factors) * V . . 

3^ For annual budget alloca^:ions, ratios of actual 

perrormance to standard at the \abor area level, would 
be combined through weighted averaging prbcedUre ' 
to yield a single performance sco^e for ©acn state - 
The state scor^e would be used in the annual funding 
allocation process, ±jx a planner sij(lilar to the current 
BEF, or fov allocating' supplemental^ resources * 

4, The method foi; calculating expected performance could 
be used to estimate the performance increases to be 
achieved thjfough optimally allocated Supplemental 
resources, by labor area, state, and/b^ national total* 

In summary, the recommended initial approach focuses upon the 
ptoblem of getting a better understanding of **what iS"^ whaft 
"factors affect the current operations of the ES, and what*ways 
can be found to 'improve those operations. It defers questions 
abQut "What byghfto be"^ but permj-ts incorporation of the 
answers to sucK questions when the answers are available in 
the future; The remainder of this volume describes the specific 
methods for developing and u^ing output and input standards, 
including alternate, methods where appropriate and the limitations 
on. the results imposed by the current stage of development* 
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chapter III 

METHODS FOR ESTABLISHING AND USING OUTPUT STANDAilDS 



IntrQduction 

1- Practical Def i;iitions ^of Out^t Standards 

Output standards have two primary uses: in the performance- 
' based budgeting "system, they can be, tlie performance norms 
against which actual performance is measured? in performance 
evaluation systems, they can seirve to identify low perform- 
ing^ a^reas where other diagnostic tools may be applied to 
isolate and correct the causes of poor performance* ES 
-Output standards were first introduced in the Balanced 
Placement Formula. The development and application of the 
BPF was guided by one central sisisumption: 

The basic consideration which h^ dictated the 
cont^Qt^- or ganiza t ion^ -and structure of the BPP 
for FY 1976 is that placement in lamployment is 
the i^entr al missdon of the Employment Service . ' 
Hence^ the formula consists entirely of measures 
of placement of job seekers and of €he filling 
of job gpenings : (BPF^- Handbook^ 'p • 11 . ) 

i ^ ■ / ^ 

Pursuant. to that assumption, the BPF for FY 1976 consisted 
of some 30 placement (output) -^^elated performance ratios^ 
reflecting quantitative and qualitative aspects qf the ^ 
placement process* For each o& the ratios, an output stan- 
dard was established - the national average for thfe ratio - 
and the performance of each State was compared to that ^ ^ 
standarrd* 

Common sense and the*ory/ howeVer, suggest that it is" not 
reasonable to expect all ES agencies to perform equally, 
because they operate under^dif f erent environitisntal conditions 
that affect performance, ^ Also, as cited in Chapter I, the 
^ BPF Handbook also suggested the neted for other methods of < 
setting state output sta^idards, including the reflection^ of 
size and environmental factors/ A conceptual framework was 
presented in'Chapte^ II where '{in theory) estimates of ES 
performance potential could be "derived from a knowledge of 
specific Objectives by examining the labor market, the job 
market and the existence of competing institutions in each 
labor area* Such an ideal model, cjould yield specific 
performance norms that would be standards for the area, 
Givon the current rcalifics it is not 'possible to develop 
such norms in the short run^ * 
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With certain assumptions, however, it i^ possible to develop 
dutput standards based on* estimates of ES output achievement 
which are reasonable^ to .expect, given economic and social ^ ^ 
conditions: ^ . 

1, Assume that output standards should be "norms" * 
or "pars" tbat are attainable by ES management 
of average ^ qua 1 ity ♦ ^ 

2* Assume that average performance of ES organiza- . 
- tions in similar labor areas and under Similar 
economic conditions is an adequate basis for, 
setting^ the nopas* . ^ ' 

3* Assume that the net effect: Jot eicternal and inter- 
nal environniental conditions on ES output perform- 
ance can be measured direc^y (without ^ complete . 
model of causality) with MjEficient accuracy and 
validity, for purposes of j^jKgluating local ES 
perfo-rmance and allocatinwresources* 

The type of putput standard which can currently be de^yeloped 
can then be stated ..as follows: 

The out^put :!3tandard le ^n expected average level 
of performance based on average t>erf ormance ,o^ - 
similar labor areas^ and a t talnafclc by ^management ^ . 

of average quality^ taking into account external 

social^ economic ^policy 'and law factors* 

* *^ 
For the future^ it i's appropriate to pursue the development 
of measures of ES potential that would lead to performance 
norms that are sounder conceptu*aily than those based on 
averages. In th^ meantime ^ the "average" is known to be 
attainable. In actual uge, for certain purposes, it may be 
desirable to set the actual "norms" at some percentage higher 
or lower than the average, i*e/, to Veflect "standards of 
excellent performance" or "minimum acceptable" level of 
performance*" Because these are based on th^ average, and 
because use of such variations depends >^htireiy ^on ^management 
purposes, the balance of the repbrt refers only to output* 
performance standards based on expected average level of 
performance adjusted for labor-market conditions and other 
factors* , / ^ 

Such standards can begin to be established using existing 
sources of data and techniques* Although th^ specific form 
of the output standards may vary somewhat depending upon the 
method used to develop standards, in" general, the goal would 
be to develop standards for each labor area for each of thfe 
indic^tor^ used in the funding^ allocation process* 

Table', lil-1 on the following page shows the FY 1974 national 
averages for each of the^perf ormance indicators used in^ the 
FY a976 BPF* To use standards in funding allocation would 
require: a^ standard for each labor area ior eaclj indicator. 



Table III-l, FY 1974 National Averages of Performance' 
Indicators Used in the FY 1976 BPF 



■ INDICATOR ■ , ■ AVERAgE. 

Quantitative 

" ^ ■ 7 ■ 

Individuals Place<i per M^n-Year Worked ^IP/MY) 139*2 

Placement Transactions per Man-Year CP/M^l ^84*3 
E4ew Applicants and Renewals as a percent of the 

tlinifeer Unemployed (AR/U)'^ 80*1 
Individuals Placed a Percent of New Applicants 

and Re^newals (IP/AR) ' 26*6 

Nonagricultui^al Opfenihgs Received as a Percent * 

of EJmployment (OR/E) - 10*0 

Nonagricultural lOp^hings Filled as a Percent of 

Openings Receilved^ (OF/OR) ^ 62*3 
Agricultural' Openings Filled &s a- Percent of 

Openings Received ^ ' ^ B6^2^ 

Qualitative , . / . 

Target Group Individuals Placed as a Percent 
of Target Group New Applicants and Renewals!: 

Veterans ' 27*3 

Minority 32*7 

Poor 37^3 . 

UI Claimants 14*8 

Migrants VfY75) * 45*8 

Women 25*4 

Handicapped 28*0 

Older Worker^ 17*7 

Yquth * 33*7 

Individuals Caunseled .as ^ Percent of New ^ 

Applican^ts and Renewals^ 5^8 
Counseled Individuals Placed -as a Percent of 

Individuals Counseled 23,1 
Openings Filled as a Percent of Openings Received, by: 

Duration: 1-3 Days . 95.0 

4-150 Days " ' 38.2 

Over 150 Days' - 53.1 

Wage Rate: Urider $2.50 71.3 

$2.50-3.39 ' 60.3 

. $3.40 and over 52.5 

Occupational Level: I 47.8 

II 55.1 

III 60,2 

IV 64.5 

V 22 88.5 
VI -79.8 



Thd implications*^ of this requirement are discussed in 
Section B witK respect to alternate methodologies. For 
other purposes, hcfWeVer, fewer l^^idicatorg can be used, 
becaus;e .previous research shows that many of the 30 BPF 
n^easure?" are highly interrelated (see Volume 2 of this 
report for details) and therefore are not, independent - ' 

indicators of performance ♦ 

Measures of ES Performance and Environmental Factors 

Research completed to date has identified a nxamber of 
exC^^yjal and internal environmental factors related to 
yariou^^easures/of ES performance* This section summarizes 
"the results ■ (presented in moife detail in Volume 2, Chapter II)* 
of statistical , analysed performed to date, presents tables 
of results^f analyses at the labor area level, identifies 
gaps in the tests performed to date, and suggests additional*- 
tests that should be perforated in developing the model* 
Statistical analysis results are presented in the tables on 
the following pages* 

An important criterion for selebting the specific recommended 
"performance measures and enviroiimental factors was that the 
data are available from existing* secondary sources — ES data 
systems or DOL publications — and do not require primary 
data collection* 

(In the research conducted to date, data were used for all 
states; at the labor area level, however, time and resource 
constraints permitted assembly of performance data for only 
80 of the 150 major labor areas and precluded compilation 
of man-year resource data, because these are not regularly 
reported to the national office for si^-state areas*) 

a* Measur6^4 of ES Performance 

Perform^ce indicators in current use, by -the ES consist* 
of lyoth '"quantitative" and "qualitative" measures, the 
latter consisting" of measures of service to specific 
"target groups, qualitative aspects of job openings 
filled , etc* The following discussion " is cpnfined tb * ^ 
quantitative measures, with qualitative measures deferred 
, *to Section^ 3*3* 

Three measUr*es of ES performance are recommended for use . 
as the key peirformance indicators* These three measures 
were selected because the indicators are not significantly 
affected by stAte br local policy with respect to applicant 
tegistration or order taking, because the data c^n be 
Verified, and because? they represent key aspects of the 
labor Vxchange function, 

(1*) Productivity; Individuals placed per man-year 

> worked (IP/MY) 

We recomrfiend this as the key indicator of the 
^ efficiency with which ES resource" are utilized, 
given aata^ available from ES data systems* 
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Table 111*2 , 

Selected ES Performance Measure^ for the First Half of FY 1975 and 
External Factors at the Labor Area Level for 80 SMSA^ and 48 Balance"^ 
of- State Areas: . 



Mean Standard Deviation 



Ferfomahce Measures^ 



\ 



Individuals Placed as a Percent of" ^ ^ 

New Applicants and "Renewajs' 22.69 ^ j^.84 

Openings Filled as a Percent of . . ^ ' " 

Openings Received 67.29 " - 10,77 



ndividuals Placed as a Percent bf the \ 

Number Unfiteployed 76.02 51.91 

Openings FilleS afe a. Percent of \ 

Nonagricultural^ Employment 3.52 ^ 2.51 



Explanatory Variables 



Rate of Growth of Employiient 

10/73 - 10/74 3.38 '5.08 
Area Employment "as a PerCtetu: of 

National Employment. 10/74 .78 ^ .77 
Area Unemployment as a Percent 

of National Unemploypent. CY74 ^78 .85 

Unemployment Rate. CY74 * 5.39 , 1.71 

■' * . * 

Percent of Employment in: V ' / 

Construction ^ \ 5.70 ^ 1.79 

' Manufacturing ^ ^ 23.48 10.10 

Government ' » 19 • 04 ' 6 . 90 

UI Claimants as a Percent of , 

New Applicants and Renewals ^ ■ 26.15 ' , 14.60 
Size Class:* 

Over 3 Million ^ , ^ .023 ;^.1'52 

Million ^ . , ' .070 .257 

Under 1 Million .531 .501 

Balance^of-State .375 .486 



^ Dummy variables* Mean indicates percent of the 128 observations in.eacjii 
size class. 
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Table III-3 



X 
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^Selected ES Perfo.rmance Measures for the First Half of FY 1975 and External Factors at 
the ]Labor Area Level for 80 SMSA^ and 48 Bslance of State Areas: ' • ' 

Correlations between PerforrDance Measures and Explanatory Variable? 

Performance Measures . ^ . 



Explanatory Variables 

Rate of Growth of Employment 

10/73 - 10/74 
Area Eraployment as a Percent of 

National Emploj^n^nt^. 10/74 
Area Unemployment as a Percent 

of National Unemployment > CY74 
Unemployment Rate> CY74 
Percent of Employment in; 

Construction 

Manufacturing 

Governnent « ^ , 

UI Claimants a^ a Percent of 

New Applicants and Renewals 
Siz€ Class; 

Over 3 million 

1-3 Million 

Under 1 Million 

Balance of State 



Individuals Placed 
as a Percent of 
New Applicants and 
Renewals 



.279 

-.066 

-.046. 
-.118 

- .150 
-.308 
.265 

-.543 

«.G06 
-.066 
-.196 
.239 



Openings Filled 
as a Percent of' 
Openings Received 



.359 
-.114 

-.039 
.016 

.173 
-.195 
.365 

-.336 

. -.085 
• -.233. 

^.442 



Individuals Placed 
,as a Percent of the 
Number Unemployed 

.328 
-.284 

-.357 ' ' 
-.497 

.262 
-.331 
.301 ■ ^ 

-.359 

■ I 
-.124 

-.181' 

-.139 

.278 



Openii^s Flllecl 
as a. Percent of 
Nonagrlcultural 
Employment 



.1440 

-.246 

-.203 
-.060' 

' .443 
-.350 
.612 

-.348 * 

-.093 
-.163 ' 
-.273 
. .397 
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Table III-4 

Matri^c of. Correlations Among Explanatory V^riablea used in the .Analysib of Eictemal Factors 
on ES Performance at the Labor Area Level for 80 SHSA's and A8 Balance of State Areas ^ 

Correlations , 



B^lanatory Variables " ^ (2) (3) ^ (Af (5) (6) (7)- (8) (9) (10) (11)- (12) 

(1) Rate o| Growth of Employment • * " 
10/73 - 10/7A -,067 -,010 .089 .300 -.252 .,422 -.091 -il30 -vl39 -.539 .670* 

(2) Area Employment, as a Percent of v ^ 

National Employment, 10/7A .9^0 .158 -.2^5 ' '.192 -,i2A ,035 .700 ,290y-.A59 .102 

" ' ■ ■ ^ . 

(3) Area Unemployment as a Percent * , 

of National Unemployment, CY7A. ^ * \A22 -.295 * .189 -.'055 .0A3 .593 .282 -.A91.172 

(A) Unemploymaitt Rate, CY7A -.237 ■ -.001 ,036 '^169 .069 .092 -.1A8 .083 

Percent of EiAployment in: " * ' 

(5) . Construction .-.A8A .A68 -,U8 -.180 .-^086 -.D68 .172 

(6) ■ Manufacturing - * * . -.A78 .251 .023 -.103 -.002 .050 

(7) Govemm^t . * -.201 -.105 -.05A -.306, .376 

(S) ' UI Claimants ae a Percent o£ . ■ 

New Applicants and Renewal© , . ' ^0A6 j;020 .013^.036 



Siafift Class: * * - - . . 

(9) Over 3 Million . ' . / -.0A3" -.165^.120 

*(10) 1-3 Million ' ■ : ^ ^ -.293-.213 

ai) Under 1 Million . . ^ I, ' " ^ - -.825 

(12) Balance of State ^ ' ^ l.dOO 
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Summary of Regression Analysis of External'Factors on Selec^d Ep Performance Measures at' 
the Labor Area Level for the First Half of FY 1975 for 80 ^SA^ and 48 Balance of State Areas 

Regression Coefficients (b) ^ ' ' . - ^ 

BETAS (B) ' * ' 

. * Percent of Variance Explained by Each F^g^or (Siv^^ 

Individuals^ Placed ' " 



Ejcplanatorv VarlatiJ.es 



to 



Rate of Growth . of Employittent 

10/73 - 10/74 
Area £inplo;^eiit a Percent' of 

Katl&nal Employment » 10/74 
Area Unemployicent as a Percent 

of National Unemployment » 5Y74 
Unemployment *Ratc» CY74 ^ 
percent ^bf .Employment In: ; . 
ConstrucMon . ' 

Manufacturing ' 
j£i ' ^ Gove^ment^ ^ . ^ 

Ul^Clalman^ as a ^Percent of 

Nev Applicants and Renewals 
Size Class: * . 

Over 3 Million 
1-3 Million 
' Under 1. Million 

Balance of Stated \^ - 

Constant 

Percent of Variance Explained (Rf) 
Standard 'Error of Estlsia^e 



*^gnlf leant at the .05 level 



as Percent of 
New Applicants and 
^ Renewals (IP/AR) 
b BrT 



_ _ ^, i ^ 

.12 .078 .022 1.11 .109 .036 

-7.D7* -.694* .0.46* -53.68*^.796* .226 

4.51* .532* .024* 26.49* .434* .155 
-1.05* -.229* .027* . -19. 07*-. 628* .312 



Openings Filled 

Individuals -Placed Openings Filled >as a Percent of 

as a Percent of the as a Percent of Nonagricultural 

^ Number Unemployed (IP/jJ) Openings Received Employment (OF/Emp)' 

- , ~b B Br ^ b ' , B ^ Br 



- .55 -.126 i.018 
-,17* -.219* .067* 
-.'.03 -.0^6 .007 

- .25* -.466* .253* 

- 9.91* .192*;.001* 

-1.63 -.104 .020 
2.36 ' .146 .035 

43.89 . J 
42.04 
6.25 



-5. 69*-. 196* .051 
-1.48*-. 287* .095 
.17 .022 * .007. 

\ . .66* .186* ^67 



87.89* .435* .079 
'8.25 .080 .011 
35.84* .336* .093 



261.41 
-59.05/ 
34.76 



ERIC 



- .03 -.014 .005 . .03 .061 .027 
-8.98* -.642* .073* -1.65*-. 506* .125* 

.12 .041 .00c 



6.34* *500* .020* 

- .64 -.102 .002 

- .47 -.078 .014 

- .13 -.122 .024 
.16 .103 .0^7 



.19* .135* .060* 
.01 .040 .014 
.13* .357* .2l9* 



- .-20* -.271* .091* -.04*-.233* .081* 

, — ^ ' A. 95* .300* .028* 

-13.10* -.'313* .Q73* 

-9.00* -.418* .119* -.97 .194 .044 
■ .51 .099 \039 



86.67 
37.53 
8.87 



2.06 
56.53 
1.73 



The number of individuads placed by the ES is the 
best available indicator of ES production and 
effectiveness, although it. does not reflect non-^ 
placement outcomes; nor does it measure long-run 
gains in employment and earnings as a resuLt of ES 
services". ' / 

ES mani-years worked in local placement and placement 
support processes is recommended the denominator 
.because it is the best available measure ox resources 
applied. ^ ^ . > ' 

^Additional research is required to develop more com^ 
plete measures of ES labor and capital, utilization 
for use in ES productivity analysis. As current prp-^ 
jects sptmsored by DOL produce more complete pro- 
ductivity indexes, these should be substituted for the 
limited measure . ... 

Service to the Labor Market:^ Individuals placed as a 
percent, o^ the number of unemployed 
individuals (IP/U) ^ . , ^ 

<t , ' 

This is a measure of performance against potential and 
is a composite of two more traditional measures* One 
traditional measure is the placement rate :(JP/AR) , i.e 
the number of individuals placed as a percent of appli 
dants available. The denominator , the number of ES 
appli-c^ts reported through ESARS / is a controllable 
number at the local office level, and therefore is 
subject tb manipulation. 

The other traditional measure, penetration of the' 
labor market, has often b^en defined as the number 
of EfS applicants as a percent of the labor force 
(AR/LF) . However, the need for ES services should 
be measured by the number of job seekers in the labor 
force, not the total labor force. The best available 
indicator of the number of job seekers is the number 
of individuals unemployed, although it is not presumed 
that the ES should se?rve all unemployed individuals , 
implying vthe ratio^AR/U*^ 

The coitqposite measure is in fact the. arithmetic 
^product of the two; 

X — = IP/U. 
AR ^ U ^ 

It l(as a verifiable numerator and an independent 
denominator. .Except fpr monthly ave^ag'es , however, 
the number jDf different individuals iinraiployed ig 
estimated only through a national work experience 



survey. RAL 802 (p. 27) suggests the following 
algorithm for estimating the nitmber of different 
individuals unemployed in the labor area; 

In estimating the numbers of different indivi<^uals " 
% ^ who will be unen^loyed during a year (in the near- 
I poverty or other categories) / the follo^ng factors 
] should be applied to estimates of average unemployment 
for the year. 

Average Unen^loytnent 

Rate (Percent) Factor 

Under 4.q 4.0 to 1 

4.0 to 4.9 3.7 to 1 

5.0 t;o 5.9 ^ 3.4 to 1 

6.0 to 6.9' ' 3.2 Jto ,1 

. 7.0 and pver 3.0 to 1 

No.teipwn method exists for estiinating the number of 
jin^ployed persons by demographic characteristics 
^nor the number of subemployed* persons 'at the labor 
$irea level on an annual basls^ as vrould b6 required 
for useviji output standards. 

Service to the Job Market; Job openings filled as 
a percent of nonagricultural wage 
and salary employment (OF/Emp) 

This measure is also a composite {arithmetic product) 
of two more traditional me^sures^ job Openings 
^received as a percent of employment (OR/Emp) , and 
openings filled as a percent of openings received 
(OF/OR). ^The coinposite measure is recommended, 
because tne reported number of openings received is 
subject to manipulation. ^ 

It would be desirable tb have a denominator that 
reflects the potential need for ES services ^rather 
than merely total employment. A desirable 'measure 
would be th^ number of new hires in the area, but 
that data is currently collected only for manufac- 
turing, which comprise^ only about 25 percent of 
employment and of ES openings filled. 

Further research should be*done to determine if a 
proxy could be developed for new hires data for 
all industries. If one ccMild assume a constant 
relationship between growth in. employment and new • 
hires across all industries or between manufacturing 
and nor^manufacturing, total new hires could be 

estimated as a function of the "growth rate of 
employment, pending direct collection of the data. 
Unfortunately, it does not appear likely that the 
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assumption --would be valid. 
Environmental Factors Affecting Performance 
a* Stable External Factors 

(1) Industry Composition 

r 

'It is often hypothesized that the higher the per^ 
centage of employment in construction, manufacturing 
and government, the lower ES performance would be 
because each of these industries has distinct hiring 
qhannels. In the analyses conducted to da^e, the 
* hypothesis tends to be confirmed for construction 
and manufacturing, but not for government. The 
results ^differ with different perfprmance measures^ , 
perhaps indicating interact-iions with^^^labor force 
composition and growth. ^ 

More important, the hypotheses in Figure II-2 suggest 
that industry composition^ affects ES performance 
primarily, through differential rates of the rates of 
labor turnover, which may have a pattern that is much 
' - d^^fferent from the percentaige distribution of employ- 
ment. Growth rates of employment by industry should 
bfe tested since they are the only proxies available 
from published sojirces. 

Measures of concentration, dispersion, and the rifin^er 
of alternate hiring channels have not been developed. 

(2) Labor Force Composition 

Analysis .results show the percentage of labor force 
ift low wage, low skil-1 occupations to be? positively 
related to ES performance at the state level. The 
relationships with other occupational groups have 
not been tested, nor lias educatiohal attainment Of 
the population. Percerit,^ in minority groups was not 
related." 

+ 

The hypotheses in Figure II-2 suggest tlmt labor 
force composition operates on ES' performance pri- 
marily through its effect on applicant composition, 
an hypothesis that should be tested by correlation 
(simple or multiple). The direct effect is through 
the mix of applicant capabilities in relation tSo * 
job requirements for whix;h there is no simple testV 

(3) Si-ze of Area 

The hypothesis is that ES performance, will be lower 
in larger areas due to the existence' Of competing 
institutions and agencies. One specification of 
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size " employment in the area as a , percent of national 
\ employment - is negatively related to ES performance* 

Variables representing discrete size ^classes were signi- 
ficantly related to ES performance at the labor area 
-level, but the specif ication used needs further testing* 
One might expect a Curvilinear relationship^ suggesting 
that the logarithm of size would be a more appropriate 
specification* .This variable might also reflect in- 
directly the effects of concentration or dispersion of 
' " emploj^ent* 

. , 

The niinnber of unemployed as a percent of the national 
number tends to be positively related to performance, 
- ^ perhaps because where more people are unemployed the * 
average level of skill, experience and allocatioh of 
/ the Tinemployed individual is higher* Since, this is a 
measure of ES potential or thd need for ES. services, 
the result -suggests that the ES does a better job where 
there^ is more need for its , services* 

Volatile External Factors 

The following factors may change dramatically over a short 
period of time, indicating the need for representation in 
a dynamic model: . - 

{ 1 ) Un emp loyme n t 

It is often assumed that ES. performance will be lower 
where the unemployment rate is high* Several t^sts* 
at the state level using the published unemployment 
rate, th^ rate of growth of th$ unemployment rate, and 
high vs.* low unemployment rates, resulted in the con- 
clusion that the unemployment rate pe :^ se dod^ not have 
consistently significant relationships to performance. 

Tests at the labor\area level show a negative relation- 
ship to the applicant placement rate and to the openings 
filled rate, but no relationship to the other measures* 
The hypothesis in Figu« 11^2 states that the fill rate 
should increase, but that assumes constant resources* 
In fact, during a peribd of rising unemployment, 
resources may be' diverted (explicitly or implicitl^^) 
to UI functions* * ' ^ 

Measures of the severity of persistence of unemployment 
published in *'Area IJrends" have i>edb been tested, and ^ 
additional research is needed to examine the interaction 
with other variables , especially policy factors such 
as UI work test and registration requirements* 

/■ — • ■ 
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Because the results were not as expected, some 
further <somment is in order* First, whi^le it is 
^riie that the unemployment rate is related to the 
number unemployed and to th^ size of the labor 
force and to the* number employed, this does not 
meam that any finding^ correlating the measures 
of size with ES performances ipso facto confirm 
/any hypotheses about the Unemployment rate itself, 
which after all is only ohe (sometimes' inappropriate) 
indicator of labor markep conditions* In statistical 
analysis, one must oftery use a related or trctns^ormed 
Specification even though it differs from the -more ' 
common statistics* Second, hypotheses about the 
unemployment rate relate to the effects of its change 
over time, and not to yche differential effects of 
higher or low^r* rates /on a cross-sectional basis. 



which* is the form tested^ 



V 



(2) GtQwth 



The hypothesis is ^that "ES performance will be higher 
where there, is a higher rate of growth in employment* » 
This has been confirmediin the statistical analysis 
at the state level, but not at the labor area level* 

Further research is needed on the components of 
growth by industry and interactions with other 
variables* The hypotheses in Figure II-2 suggest jthat 
growth reduces the relative supply of qualified 
workers but increases the 5(^emand;f6r them* The sep- 
arate effects should be tested*'* 

Internal Environmental Factors 



Internal environmental factors are those factors that are 
mare or less under the control of • the en^loyment security 
system a^t the. federal or state level, but'not at the locai 
le^l. ^ , ( , 

(l)JU^ Policy and Lav? Fat^tors 

..^y- is hypothesized that' policies- and laws such as those 
Bjhat require the ES to perform enforcement activities, 
provide work tests,' and list openings from federal 
rbntractors will result in lowerj ES performance because 
ES resources will be expended in: registering applicants 
or listing jobs tfnd attempting to serve applicants and 
employers who may prefer not to fase the ES* Resources 
so expended are diverted from se^tvice to those 
applicants and employers who wanjb Es services* 
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If state policy or law requires *all claimants to 
register with the ES, then the ES will be consuming 
resources in efforts to register and^to place claimants 
who are not active job seekers at the time they file 
claims, although the motivation to seek new employment 
would be expected to increase with the duration of un^ 
employment and particularly with the exhaustion of 
benefits* . 

t * 
In statistical tests, claimant new applicants and 
renewals as a percent of all new applicants and renewals 
was tested statistically and was found to explain 44 
percent of the differences among the states in pro- 
ductivity in FY74 (a period when unemployment first fell 
then rose) . It was in fact the single most important 
factor in accounting for differences in produptivity 
among the states when external economic conditions were 
held constant (statistically). Similar measures of the 
effects of welfare and food stamp registrations were 
not statistically significant, but their effects need 
further examination* 

These factors are in part external to the local ES, 
\>ut are amenable to influence and change at the state 
or* federal levels. 'Considerable additional research is , 
underway, to devise^appropr iate measures of policy and 
law factors, to estimate their effects on performance, 
and to develop suitable policies with respect to in- 
corporating the factors into a model for setting output 
standards,. 

Resources , 

The statistical model tested to date did not include 
any measures of ES resources as variables to explain 
differences in output performance among areas, but it 
is^ intuitively «Sbvious that the level of resources has 
^a bearing on ES perforfttance as measuited by the 
recommended indicators* The following hypotheses, 
suggested for testing in future analyses, each attempt 
to.rela1;e i;he measure of resources to, the measure of 
performance in ^ way that corresponds to the hypotheses 
in Table II-l. Because in each case the hypothesis is^ . 
that there are decreasing returns to scale, the resource 
measures are specified in a quadratic form, holding 
a,ll other factors constant. 

'V - ' 

(a) Individuals placed per man-year worked decreases 
at an .increasing rate as the numbeavof man-years 
incifeases. T,he explanatory man-year variables 
would be spedlfied as total man-^ears and as the 
logarithm of^ total man-year&. 
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(b) Individuals placed aa a percent of the number un- 
employed Inqreases ar a decreasing rate as the 
n\unber of man-years per 1000 unemployed increases. 
The e:^plarifetory vajriaoles would be ES man-years per 
1000 unemplpyed and tti^ logarithm thereof. 

(c) Openings filled as a percent of employment increases 
at a decreasing rate as the nuinber of man-years 

^ per 1000 (or 10,000) emp^yment increases. The 

, explanatory variables woMld be ES man-years per 
lOOOj (or 10,000) employment and the logarithm 
thereof. y ' ■ / 

. * 

The implication of the logarithjy.c term is that out- 
put aOTieved via a small increment ''of resources is 
^* less than the averse output of ill resources. 

B. Methods for Establishing Output Standards ' - . " 

The basic framework within which output ^r^ placement standards 
will be developed is a model for computing Estimates. , of 
expected ES performance, using statistics^tr^at taJce into 
account the influence of external factors oi^ ES per^formance 
" across labor areas. The end products would }pe a published 
list of the standards for each of the 150 ma^or labor areas 
and for balance-o£-state areas; the methodology for calculating 
the standards; a computerized method for updating the standards; 
arid computerized tools for using the model fon each of its • 
purposes. . * * 

" ■ " / ' ' ■ ■ ' 

The establishment of output standards for ES performance 
poses a number of methodological problems. Wij^h respecp to 
each, there are both technical issues, relating to the best, 
state-of-the-art techniques for estimating the^^tandards , 
and political issue's, ireiating to the degree o'f^ difficulty 
in interpreting the methods used so that the non-technici,an 
can intuitively understand and trust the results to be fair* 
and equitable. 

This section presents the following topics: . ^ 

\* Alternate methods for establishing standards for 

quantitative performance indicators in a static. model ^ 

2, The need for a dynamic model 

3. A method for setting standards for qualitative 
indicators' 

4* The problem of 'standar<Js for sub-parts of labor areas 
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Previous research'^ and the .needs for additionaj. reselTrch are 
discussed under each atopic. 

1. Alternate methods for establishing static output standards a 

Two alternate methods for establishing output standards 
have been considered and partially tested'. Ope method 
uses muljtiple r^gres^on analysis to estimate an equation 
for adjusting average performance , which can be ilsed to 
set an output staridard for each pextormance indicator for 
each unit of 6bservation (labor area) without explicit, 
grouping of labor areas into types or classes. The other 
method uses factor analysis to identify significant 
external factors, and a hierarchical clustering technique 
to group labor areas into discrete classes based On the 
external factors. Each approach has both advantages and 
disadvantages for the proposed applicatipn, and th^y will 
be discussed in turn. 

a. Adjusted performance equation 

Development ofjan equation to adjust performance 
averages for external factors tfiat affect performance 
requires the specification of hypotheses relating eacfi 
of the external factors to each performance indicator, 
creation of a data base containing both the performance 
indicators and the external factors (appropriately 
specified), and us^e of nvyltiple regression analysis ^ 
/'to estimate the coefficients of the prediction equation 
for each of the performance indicators. 

+ 

The multiple regres3:ion analyses results in aii 
estimating equation, which indicates how much the 
performance measure is .expected to increase (or 
decrease ) as a result of a one--point increase in 
the external factor. The equation can be used, then, 
to compute an expected value for the performance 
measure for each specific labor area and fpr each 
balance of state area in the equation. 

The expected value forigsreach area is, by definitic^n of 
the statistical proces^s, based implicitly on the 
national average for that pei;formance measure, adjusted 
for differences betwe^ the external factors in the 
area and the national average for each factor. Each 
* expected value also has a st;andard error of estimate 

which defines the level of precision of the^ estimate ^ 
and which c^n be employed whenothe expected values 
are used to set output standards or measure performance 
against the standards. The standard error of estimate 



r4presents the area of ignorance about factors affecting 
performance , Hope fully , the principal component ' is 
the variation in ES management among the areas^ although 
measurement error^, specification error and ^eluded 
factors also conttibufe*^ ' . 

The research completed to date has tSsted only one 
method Vfor setting output standards - a linear 
equatiori for calculating expected performance, 
wherein the coefficients of the estimating equation 
Were estimated using* regression analysis* The 
equation was of the standard form: 

Th^ table on the following page shows two exaiaples ; ^ 
of Ijow expecteid performance can be qalculated using 
this type of equation* The equation coefficient 
multiplied by the actual value of the factor for an 
area yields the fatctor^s contribution* to expected 

performance for the area. The algebraic sum of contri* 
batioreplus the constant is tHe expected performance* 

The particular estimating equation wa^. subject to the 
limitations that a linear relationship- was assumed 
between each of th« explanatory variables and the 
dependent variable; no interaction terms were included, 
the explanatory variables did not include any data on 
labor force composition, and only 80 of the 150 
largest SMSA's were included in the estimate* 

If output 'Standards are to be set using a performance 
adjustment equation, several additional research 
steps are required* 

(1) . The assumption of a linear relationship between 

the ES perfotmance measures and the external 
factors needs to be examined to ascertain whether 
othef specifications of some of the pxternal 
factors (etg*, a logarithmic transformation of 
the size measure) might not be more appropriate* 

(2) Interactions among the-j^ternal factors need 
to be examined and perfSips included explicitly 
in the equation* For example, industry composi* 
tion variables, partitrularly the percentage of 
employment in manufacturing, may have more of 

an effect on ES pefformance in combination with 
growl^ rate than as a separate variable* 
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. Tab'l^ III-6 

Example of How Expected performance is Confuted for One Measured 
Individuals Placed as a Percent of the tfumber Uneirtployed (IP/U) 



F'^uation Coefficients 
(1) ' 



Actual Value* 
of Factor 
(2) 



Contribution to 
Expected Perfpmiance 
(3) = (1) X (2) 



Actual Value Contribution to 
of Factor Expected perfdrmance 
(4) . (5) = (1) X (4) 



4. 
c 



constant " 261,415 
Growth * 1-106 
% Kmp, ' -53,678' 

% UneiDp. 26,494 
Uneitp* Rate -19*074 
6 Construction '-5,687 
7, Hanufacturing-1,485 

8 Government , 172 

9 Claimants -*661 

10, Size Class 1 87,893 

11, Size-Class 2 8,253 
12 Size Class 3 0,0 
13, Size Class 4 35.838 



*,0727 
-3489^ 
.1085 
2,0664 
6^7926 
19,6876 
20,6684 
21,8591 
,0 ' 
,0 
1,0 
,0 



261,415 
,080 
-18,728 

2,875 
-39,414 
-38,626 
-29,234 

3,554 
Tl4,438 
,0 

! 

,0 



-1,5511 
^ -4102 
. ,2635 
3,7725 
4,9120 
19,6478 
19,0918 
15,2522 
,0 
,0 
1,0 
,0 



261,415 
-1,715 
-22 4P19 
6,981 
-71,955 
-27,3*32 
-2?>i75 
3,283 
-10,076 
,0 
,0 
,0 
,0. 



Expected performance; \ 127,483 108,809 

A^-^-ual performance: 163,044 ^ ^,791 

Difference ; ' -35 , 36 44 ,018 



S^jndard Error of Estimate: 34,757,7 
Percent of Variance Explained: 59,05% 



Notes t * Column (1) Lists the actual coefficients of the adjusted equation estimated 
using regression analysis. For purposes of illustration, all explanatory 
variables were included in this exaiDple, even though some wer^e not statistically 
significant. The constant term is by product of the statistical process and 
has no independent significance in this application. 

Columns (2) and (4) List the actual values of the explanatory factors for the 
two SMSa's in the exaitqple. 

Columns (3) and (5) List the contribution of each factor to expected 
-performance. The values are con?>ut€d by multiplying the actual values 
of the external factors for each area times the equation coefficients 
in Column CD , ^ * 

Expected Performance is calculated by (algebraically) summing the 
contributions of each factor in Columns (3) and (5) respectively. 

Actual Performance is the IP/U measured for each area. 

The 'Difference is Actual minus Expected IP/U. * >^ 
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(3) The model is incomplete without any reflection 
of labor force con^iosition* Occupational group 
(low wage, low skill) was found, significant in 
research at the state leviel, while percent of 
labor force in minority groups was riot; neither 
variable was tested at the labor area level. 
Even though data ^on labor force coir^psition 

is available only from the 1970 Census, it is 
possible that changes in composition Over time 
. are slow enough that use of 1970 data would 
improve the overall model* 

(4) Only one variable reflecting policy and law has 
been tested at the labop area level - the percent 
of new applicants and renewals who were UI 
claimants. This factor was found to have a 
significant negative effect on ES performance at 
both the state level and the labor area level, 
even when unemployment levels and rates* were 
included in the same analysis. Analogous 
variables for Food Stamp and welfare recipients 
were tested at the state level and found to have 
no significant effect on productivity, but should 
be tested kt the labor area level. It is 
necessary also to examine the effects of manda- 
tory job listings on job market penetration and 
percent of openings filled* 

(5) Research to date has focussed on annual average 
performance as affected by annual averages, of 
the external factors. In reality, the effects 
of intertemporal changes in such factors as 

^ unemployment rates and growth rates may impact 
on the ES over shorter time periods and/or with 
lags, so that the effects are concealed by the 
use of annual averages* These dynamic, time- 
dependent effects need further examination* 

The advantages of the performance adjustment 
equation are: 

(1) A specific standard is computed for ^ch defined 
labor area for each performance indicator, using 
the actual values of the external factors in that 
area to adjust* 

(2) A standard error of forecast can be. computed for 
each area, using the standard error of estimate 
and the standard error of the coefficient. 

(3) No explicit grouping or classification of areas 
is required, imni^aizing the' scope of negotiation 
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on the part of ES administrators td have areas 
within their states^transferred to categories 
having lower standards. 

(4) The resultant standards are based on the adjusted 
average performance of (implicitly) similar areas, 
and reflect what should be an achievable average 
performance • 

The disadvantages arei 

(1) While the estimating equation itself is a 
relatively simple arithmetic expression,^ the 
process by which the coefficients of the* 
equation and the standard^errors are computed 
is compleJc. It may be difficult to explain the 
process with sufficient clarity to administrators 
to permit them to have confide^ice in the results. 

(2) ^ Eac^h performance indicator recjuires a separate 

estimating equation. If the current BPF iftdicar 
tors are used, 30 such equations would need to 
be 'estimated, and 30 specific standards would be 
calculated for each labor area. Although con^jutei;^ 
could do these calculations quick^Ly, actually 

using all^the measures could be ^ expensive 
proposition. The process could be simplified 
through use of a composite measure combining 
all performance indicators, but this would 
sacrifice much of the detail needed for 
Operational evaluation and diagnosis at the 
local level. 



(3) It is likely that a given external factor will 
be related to the (different "performance indica- 
tors in different ways. For example, the growth 
rate may be related to productivity and to jobs 
filled^ but not to' applicants served. Again, , 
this may be difficulx to explain clearly to ^ 
administrators. 
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(4) The adjustment equation is 
which tends to imply causal 
factors and ES performance, 
the causality linkages are 
relating ES internal proces 
and constrained by external 
would reflect only the gros 
constraints, with the inter 
ness implicitly reflected i 
or estimate. ' - 



written in a form 
ity between external 
In fact, of course, 
complex function- 
ses to measured outputs 
factors. The equation 
s effects of the 
nal process effective- 
n the standard error 
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Grouping of labor areas^ " ' , ^ 

The second method for setting output standards is ■ 
based upon a classification schema or typalogy that 
can be used to grd^up la^r kreas into a limxted 
nunsber of homogeneous ^gifoupings. JSfithin each such 
group or type, the output standards would be based 
upon the average performance of all areas for pach 
of the performance 'indicators. The project plan 
called for performing the statistical analysis' needetj 
to -provide- an initial indication of the kinds of 
groupings and standards that could be developed, but 
technical problems and resources limitations prevented 
the planned tests. Therefore, the following material 
discusses the principles involved and provide? 
-hypothetical example^. 

The *ey step in grouping is that of classifying labor 
areas into homogeneous groups,^ OnC^ again', the' 
hypothesis is that external economic Conditi6ns impose 
constraints on ES performance, so the cjbject^ve is to 
define group's ^f areas which are homoge^^eous with 
respect to the relevant, exter;nal factors. If there 
were only two re^levant factors-, each of which had 
discrete, .values of '^tiigh" and "low", then 4 classes 
would be sufficients ' ^ ^ - 

ClaSs J:. high-high ^ 

. i Class 2: high^low 

Class lov^-high 

Class 4j 1ow"1ow ' o ^ 

However, the research completed to date has identified 
several significant factors, most of which are con* ^ 
tinuous variables. Tests of discrete specifications 
of the unemployment rate* did not yield statistically 
significant results, although d^,screte size class 
variables v^ere significant in one test at the, labor 
area level.? Further examination is required. 

It appears, thln^ that, the bes»t method for developing 
the groupings is to ude a combination of factor 
analysis and hierarchical clustering. The factor 
analysis takes as mtany descriptive variables as one 
wishes to test an<f selects those that have theV 
greatest independent v^lue for discriminating among 
the labor areas. (It identifies a minimum number of . 
orthog^^l > vectors in multi-idimensional space^^ then * 
selects the observable factors most nearly correlated 
with each such vector 1) 
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The hierarchical clustering technique^ takes the factorls 
thus identified and creatfes groupings of areas so that 
the areab within a group are most similar when all 
factors are taken together and are most different from 
a^eas in other groups. The tnechnique is hierarchical 
beca'lis€ it starts with each area as a single groups ^ 
then proceeds stepwise, to combine areas into, fewer 
groups until asLl areas have been combined to one group* 
The number of groups used is essentially arbitrary ' 
with respect to the clustering technique and is chosen 
based on the practicality of the application\at Jiand* 
For purploses of developing output standards^ it seems 
reasonable ^to select -the step 'in the process when all 
areas have been combined into To or 12 groups.. Within 
each such gfoup^ standards would be ^et by calculating 
the average of each performance indicator within each 
group. The table on the following page shows hoVr / 
such standards might be presented for three arbitrary 
types of labor areas., . ^ - 



Grouping iareas using a combination of several factors 
can result in difficulties in interpreting the results 
to administrators* The map btg^low shows how three dif* 
ferent types of labor areaS might compare on ajiy^two 
factors* Some of the ar^as in Group A are more similar 
in size to areas in ^Group C than to other Group A^areas 
while- Group B areas^'are similar in industry composition 
to some Group C areas* If Group A or' 5 have a higher 
output performance standard than' Grou^^ there would b 
the appearance 6f an inequi^ty* 



SIZE 




Tilt 



Industry 



e advantages of using groupiji^s of labor areas to 
set . standards are: 



(1) Each area is classified" into a specific group. 

The list of areas in each group can be published 
so .that administrators will know with whom they 
are being compared, ' * , 
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Table III-^. Example of Output Standards Based on 
Averagea Within Type of Area for 
Foiir .Performance Measiires 



P^'rformance Measure 

OF/E 

IP/AR 

OF/OR- 



Type A 
113.92 
2.32 
27.20 
57.43 



Type B 
38.03 
1.68 
14.74 
54.85 



Type C 
28.36 
1.47 
13.18 
53.89 



{Target Groups) 



{Typtes of Jobs) 
etc. 



J 
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(2) The output standard is based on an explicit 
average of ^11 areaa within a given gfbup for 
each performance indicator t The standard 
deviation for each indicator within each group 
can also be- computed and applied in different 
us^s of output standards t 

The disadvantages of using this approach are: 

[ ^ ^ 

(1) Because the groupings are established usin^^ 

many factors, it is difficult to ^explain 
precisely why a given area Is in one group 
rather than axiothejTt 

(2) A given area may be more similar on one or two 
factors to certain areas in different groupings 
with lower standards than it is to other areas " ^ 
"Within the same grouping on the selected factors t 
This leaves an apparent^^aotential for inequities 
and room for negotiation and dispute, 

(3) Even when all factors are taken together, certain 
areas will be on the margin of the group in which 
tliey are classified, which means th^t they will also 
be near the margin of the next higher (or lower) 
group. 



(4) It would be desirable for the groupings to remain 
relatively constant over time, but certain 
potentially relevant: variables such as the 
unemployment rate are volatile. Therefore, 
either the labor areas must be regrouped each, 
year to reflect volatile factors, or a separate 
adjustment to the output standard must be made 
for the volatile factors. 

c. Recommendation 

Wo firm conclusions or recommendations can be made 
on the choice of a method until the two methods 
* have been subjected to further testing on their 
technical and political merits. 

Toward a dynamic model 

The preceding methodology will result in a static model 
which calculates expected ES performance based on total 
performance for the year (or other time^period) and the 
average of external factors for the s^me period. Such 
a model is quite useful for a variety of purposes, but 
it cannot fully reflect the real world, in which events - 
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oecur in a" time continUiam and in which current ES ^ * 
perfontiance Is a functioa rfot only ofc current, conditions 
but- also of recent history/ In the real world, use of 
annual data .may ^obscure time-dependent relationships 
between ES performance and external factors if the impact 
of a change^ in external factors such as rapidly rising 
tinempToyment passes within one or two quarters. The 
effects can plainly be seen by an observer in any ES 
office, but ntay wash out in annual averages. 

The purpose of a dynamic model, then, is to provide a ^ 
framework reflecting the instantaneous and time-lagged 
effects of cyclical variations in external factors on 
ES performance, and in particular t)n individuals placed 
per maji^year. To develop the model, it is necessary to. 
explore £Bese interten^oral cyclical relationships, 
adjusted for seasonal factors and for variations" caused 
by the structure of the ESARS reporting system. 

Quarterly ES performance data are affected both by true ^ 
seasonal factors and by artifacts of the reportijig 
systems. ESARS is a cumulative reporting system, 
which counts each individual placed (IP) only once 
during the -fiscal year, no matter how many times the 
individual is placed. The annual. ratio of transactions 
(P) to individuals is about 1.5. It appears that 
approximately 75 percent of the individuals placed 
more than once during a fiscal year usually in 
casual, day worker, or other short-term jobs, are 
placed i^^ the first quarter of the year. Thus, even 
though the level of placement transactions may remain 
constant throughout the year and. the cumuljitive 
total of individuals placed year-to-date . will rise, 
estimates of the net number of new individuals placed 
will always show a qaarter-to-quarter decline. 

Unfortunately, there does not apt>ear to be a single 
metljod f^r estimating the quarterly effects of the 
reporting system in isolation from seasonal or 
cyclical variations^ 

A dynamic model woulcj represent a^ very significant 
step forward -In refining the ES management system. 
Some of the dynamic effects bf cyclical change on 
ES performance might ultimately be reflected in "the . 
annual ftinding allocation process, insofar as the 
data on external factors found relevant are available 
shortLy after the conclusion of each quarter. These 
effects could- also be reflected in specific resource 
requirement^ ^and performance projections which are a 
part of ^the annual budgeting process.- Given the 
current" needs for further development of the static 
model, however, it is recommended that any development 
of a dynamic model be deferred. 
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Developing standards for qualitative performance indicators 



At the beginning of this chapter, it was recommended that 
output standards be developed for 'three key quantitative 
ES performance indicators. One of these is admeasure used 
in the BPF, while each of the other two is a coirposite of 
two BPF measures. In additiot^-f^t isr desirable for many 
reasons to have also output standards' for quality 
performance indicators relati^ng to types of ES apprlicants 
placed and types of lobs filled. Whtle the analysis 
presented in Vplume 2 of -the report shows a high degree 
of interrelationship among the 30 BPF measures and raises 
technical questions about the n^d for all of them, the 
fact remains that virtually all the qualitative measures 
in Table III—l are grounded in specific^ laws or court 
decisions (as in the case of target group placement rates) 
and/or reflect a desire to build incentives to improve the 
overall level and quality of ES performance. 

The major substantive problem is that all of the qualitative 
measures in the BPF relating to placement of . target groups, 
counseling and types of jobs filled havp denominators that 
are subject to manipulation by the ES at the local level, 
(irrespective of the method used to set output standards, 
it would be technically possible to compute a standard 
for each of tl^e qualitative measures, although it might 
be costly^) For exaitjple, a standard ^r a target group 
placemerrt rate , once set, coui^d easily be achieved simply 
by reporting as registered only' those applicants who were 
actually placed. It seems wasteful to encourage this type * 
of manipulation, especially when it might result -irl a 
lowering of overall performance due to the absence of 
application cards for job seekers and other docuirients 
needed at the operating leve 1. 

One possible alternative is to derive standards for the. 
qualitative vindicators from national averages of indicators 
that are coirponents of the quantitative measures. The 
example of target group placement rates can illustrat'e 
the method {Table III-6): ^ 

a. Calculate the Rational average placement rate 

{individuals, placed as a percent of new applicants 
and renewals) for all applicants and for each^target 
group... In FY 74, for example, the national average 
placement rate for all applicants was^26.6, for 
veterans it was 27.3, and for claimants' it was 14.8 
(Column 1) - 

hi Calculate the ratio of ^ach target group placement 
rate to thfe placement rate for all applicants* The 
,FY 74 ratio was 102.6 for veterans and 55.64 for 
claimants (Column 2)* ' 
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c. In each labor area, establish target group placement 
rate standards by applying each national ratio {of 
targe1^ group placement, rat$ to total placement rate) 
to the placement ratevfor all applicants in the labor 
area. Using data from Table III-l, Table 111-^6 shows 
placement rate standards for two hypothetical areas 
having placement rates for all applicants of 20 percent 
and 50 percent respectively (Columns' 3 and 4). 



In the following example, it is assumed that Area B has under- 
reported new applicants and renewals to ihorease its total 
placement rate to 50 percent, an unusually high number. 
The proposed method yields target group- standards ^which are 
alsp unusually high, so that unless the Area B has a 
balanced performance ,it will- ^ot be rewarded. .;s. 

The method encourages balanced placement performance without 
rewarding the manipulation of the number of new applications' 
and renewals reported, either for all applicants or for the 
target groups. Any potential Reward for manipulation is 
offset by the fact that many applicants fall into two or more 
of the target groups, so that under-reporting, new applicants 
oi: renewals in any of the larger target groups (veterans, 
minority^ claimants or women) will, Have effect on the total 
number of new Applicants and renewals. In addition, the 
actual standards would be based on historical performance, 
notv projected performance, so that precise standards would 
be available as a performance evaluation t^ol, but not as 
a target, 

^ similar method could be^applied to the development of 
standards for. types of jobs and other qualitative factors 
' the BPF , * " * 

Setting Output Standards for Labcft: Areas that Cross State Lines 

In gejjeralv it is desirable to establish and apply output 
stslndards for.ES serviced to an Entire labor area, because 
by definition each such area is an integrated economic and 
sotiaj* system. Use of the standards should encourage ES 
management to lo'cate its offices and utilize its resources 
to be of maximum service to, the labor area, and should 
reward supe^rior output performance independent of input 
location and utilisation considerations. 

Howevfer, of the 150 major labor areas for which unroiploy- 
ment rates are published in the annual Manpower Report of 
the president , 23 are SHSA's having parts in more than one 
state. This poses a particular problem when using output 
standards for allocating resources to states. In many of 
these are^s, the central city is wholly within one state. 
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Table 111^6, Derivation of Target Group Placement 
/ Standards for Two Hypothetfcal-Areas 







ttational 


Data for FY74 


Placement 


Standards 






Placement 


Ratio to All 










Rate 


Applicants 


Area A 


Area B 


Target Group. 




(1) 


(2) 


(3) 


(4> 


'All Applicants ^ 




26.6 




20.0* 


50.0* 


Veterans 




27.3 


102.6 


. 20.53 


51.32 


Minority 


32.7 


122.9 


24.59 


6U47 


Poor 




37.3 


140.2 


28.05 


70.11 


Claimants 




14.8 


^ ^5^.64 


11.13. 


37.82 


Migrants 




45.8 


172.2 


. 34.44, 


86.09 


Wonts n 




25.4 


95.49 


^ 19.10 


47.74 


Handicapped 




28.0 


105.3 


21.05 


52.63 


Older Workers 




17.7 


66.54 


13.31 


33.27 


Youth 




33.7 


126.7 


25.34 


63.35 




^Assumed actual placement rate for each area. This is multiplied by the 'ratio 
In Col. (2) to calculate the target group standiards. As an example: 
Area A Placement Standard for Veterans » (20.O)X(lO2.6) = 2O.53 

\ 



ERIC 



47 

44 



/ 



with suburban and rural portions extending into another 
state. To take the archetypical case of Washington, D.C., 
while the ayea is an integrated labor area, Maryland and 
Virginia contain all the suburban and rural portion of the 
SMSA's, while D.C.has only the central ^ity. It seems 
unreasonable to apply thfe same output standards to the 
entire SMSA, particularly for purposes of funding allocations, 
even though the entire metropolitan area is covered by a 
single Job Bank operation. 



Therefore, it is necessary to develop a further disaggrega- * 
tion for output standards, reflecting the sub'-divisions of 
SMSA's. Performance data is regularly compiled for each r 
state's portion of each SMSA, and, furthermore, for each" 
cost center in each SMSA. Each cost center can be identified 
as located in ^the central city, the. urban fringe, or the 
rural portion of the SMSA by examining its location on a 
map. Using these indicators, specific adjustment factors 
can be estimated to reflect the' effects on performance of 
location of cost centers within an SMSA using either of 
the alternative methods for setting standards. In the 
adjustment^ equation approach, dummy variables would be 
included to obtain differential coefficients? in the grouping 
approach, the difference between the average for all areas 
within the group and the average for cost centers within 
each SMSA 'part would be calculated. 

These adjustment factors x^fOuld indicate the performance 
differentials between each SMSA part and the overall 
average. They would be used in the fundings .al location 
process t3 set a standard for the SMSA parts in each 
state portion of a multi-state SMSA. For other purposes, 
they might be used as indicators of optimum office location. 

It is important to note that, while data on external factors 
can be estimated for the parts of the SMSA in each state, 
this should not be done ^be^cause the external factors 
influence the integrated whole. 



Usingoutput Standards ^ - 

The use^ bf output standards have been discussed in generalities 
previously in this report* This attempts to state in some 
detail how to compute the ^various statistics needed for each 
type of application, assuming the methods for setting have 
already been established and, that there is a need t6 use them 
for each purpose. 

1". Using output standards ass^ a management tool* 

It is assumed that es management at all levels has strong 
incentives to improve overall ES performance, as measured 
by available data on placement outputs. Performance 
improvement plans must ultimately have their effect at 
the labor area or local level, where ES services are 
actually provided and where outputs are achieved. The 
' outgut standards permit the identification of individual 
areas where actual performance is below- expected or 
standard performance. While it is likely that performance 
could be improved in any labor arga, it seems logical to 
focus on those areas that are furthest below a standard 
level of performance. 

The comparison of actual with standard performance can 
tell sl^ate ES management which ar^eas within the state 
have the lowest -perform^i^ce . If most areas within the 
state have performance below the standard, then the 
process of diagnosing the causes of poor performance 
^ shoul-d start at the state level, with a review of manage- 
ment techniques, training approaches, staffing methods, etc* 
At the labor area level, the location of offices, aissign- 
ments of staff , and overall manner in wlj^ich staff resources 
are organized, trained, and managed should be examined to 
isolate specific areas that need improvement. 

Table III-2 the following page displays expected and 
actual performance for four performance indicators for 
six SMSA'St ' These have arbitrarily been grouped by high, 
medium and low expected or standard performance level* 
Table entries for each area show expected levels of 
performance for each measure, actual performance, and 
the ratio of actual to expected performance* If all 
areas were in^ the same state, the state administrator 
might set the following priorities for further analysis 
and performance evaluation; 

a, SMSA2 would receive the highest priority, because that 
area's performance is lower than the standard for* 
all four measures, while SMSA4 is low on thrqe of 
the four measures t 

b* SMSA3 might be given the next priority, because its 
performance is lOw pn both placement measures* 
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TABLE III-2 



Comparison of Actual Performance with Estimated 
Expected Performance for Four Performance 
Measures in Six SMSA's 



\ 



r * *■ 



High Standard 
SMSAl 



Individuals 
Placed as a P^t~ 
Cextt of th« 
Number 
Unemployed 



Openlag^ F01«d 
as a Percent of 
Nona gric ultur al 
Employraent 



IndlylduaLs Ftaced 
as a Percent of 
New .AppUc^ts 
and Renewals 



Expecrted 


127.48 


3. 


85 


24 


.97 


Actual 


163,04 


2. 


55 


34 


.34 


Actual/Expected % 


127.89 


66. 


36 


:i37 


.54 


SMSA2 












Expected 


108.81 


. 3. 


43 


26 


.02 


Actual 


64.79 


2. 


09 


20 


.07 


Actual-Expected % 


49.55 


60. 


96 


77 


.11 



Openings FiUed 
as a Percent of 
Openings 
Received 



66.94 
61.39 
91.71 

68.27 
53.48 ^ 
78.34 



Medium Standard 



SMSA3 

Expected 55.91 '.97 

Actual , 61.88 2.43 

Actual/Expected % 110.68 250.52 
SMSA4 

Expected 63.84 3.37 

Actual 57.02 2.28 

Actual/Expected % 89.33 67.62 



18.01 
24.15 
134.09 

23.87 
18.41 
77,10 



59.45 
55.16 
92.78 

65169 
70.75 
107.70 



Low Standard 



SMSA5 

Expected '■ 12. 85 2 . 06 

Actual - 35.92 1.57 

Actual/Expected % 279.58 76.50 
SMSA6 

Expected 11.41 .22 

Actual ' 20.81 1.36 

Actual/Eypected % 182.41 621.03 



11.86, 
13.62 
114,95 



12. -6 2 
12.74 
100.98 



56.94 
48,74 
85.59 

57,08 
59.05 
103.46 
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c.^ SMSA's 1 and S are low on both measures of openings 
filled, which taken together with'the performance 
of SMSA's 2 and 4 may indicate a general weakness 
in sei;vice to employers in this hypothetical state. 

Furthermore, within the labor area, analysts diagnosing 
the causes of poor performance must review tlie jnanner in which 
such policies as performing the work test have. been inplemented 
par^cularly with respect to thos^ items known to have a 
negative effect x?n performance. One state found that, even 
though state policy does not require registration of all ' _ 
claimants, there were sizeable variatjjons within the state 
as to th/e percentage of new applicants and renewals who 
were claimants that* did not appear to be related to 
differences among: the areas in unemployment rates or levels. 
(The differences were perfectly rank^correlated with 
differences in performance.) 

The key point for administrators to remember is that the \ 
output standard set for a given area takes into account * 
the observed external factors - economic and social, policy 
aijd law. The observed measures of policy and law factors 
in a given area iftay, in and of themselves, indicate that 
failure to conform to the intent of a policy or law is 
having a major negative effect on performance, and correctio'n 
of the deviation will improve performance. It is true that 
correcting the problem might raise the output standard 
used for resource allocatic^n (depending upon national 
policy). However, it is also true that it is the 
responsibility of each state to effectively serve the 
greatest numbers of job seekers with the resources it has 
available. Explicit inclusion of policy and law factors in 
tfee performance evaluation and analysis process provides 
explicit guidance for the analysis of policy impleirfentation. 

Using outj^t standards in annual resource allocatiofis. 

In- lieu of using national averages as the standards against 
which performance is measured, for annual resource allocations*, 
output "standards would be set for each area, including both 
the 150 major labor areas and balance*of-statfe areas. 

It is recommended that all known factor's external to the 
local ES be taken into account in the process of setting 
output standards, including policy and la^w factors / so that 
differences between the output standard ^nd actual performance 
can reflect as nearly as possible the effects of factors 
internal to the ES (i.e., under ES control), such as location 
-of offices, overall quality of manageinent and staff, local 
office organization, etc. These internal factors can then be 
addressed by ES management to develop performance improvement 
plans. A number of problems arise, however, in interpreting 
and using, the output standards for different" purposes when 
policy and law effects are taken into account. ^ * 



To illustrate the problem o^ taking policy and law factors 
into account in setting output standards, consider the 
effects of policy with respect to the registration of UI 
claimants- In a pferiod'of rising unemployment, job losers 
constitute the largest Component of the growth in unemploy- 
ment. Most job losers are covered by UI, and most become 
claimants. "In'this context, it should be noted that t^ie 
number of workers receiving unemployment compensation as 
of November 1974 was practically the same as the number 
of job losers...*' ( Manpower Report of the President , 
April 1975, p. 29.) Many joJ3 losers, hoyrever, are on 
temporary layoff or beliqve themselves to be; they expect to 
be recalled to their previous jobs, and they are not 
seriously interested in finding a new permanent job. 
(Unpublished data from New Jersey indicate that during 1974 
only 44 percent of initial claimants were looking for a 
permanent new job.) ' ^ ■ ' 

Since UI regist'ration policies (and, to some extent, laws) 
are subject to influence by state ES a^d UI agencies, it 
does not seem desirable to lower output standards by the 
total amount indicated by the adjustment equation, which in 
the form shown in Table III-6 reduces the standard for IP/U 
by ^661 points for each 1-point increase in claimants as a 
percent of new applicants and renewals. This is partitrulariy 
pertinent for arsas where mandatory registration results in 
claimants being 60 percent or more of new applicants a:rid 
renewals, which would reduce that standard by 39 percentage 
points. In the ^bsence of " a more sophisticated procedure, 
one might set a policy that the claimant percentage above 
some fixed standard would be ignored in setting output 
standards. Based on the average of all states (about 
27 pe^rcent) and review of some specific states, a reasonable 
cut-off might be 30 percent, but further analysis is 
needed before policy could.be set. 

1' 

Since ES resources are allocated to s^^ates, not individual 
areas, the performance of the individual areas in relation to 
their individual standarxis must be combined to arrive at a 
single state summary score, similar to the summary score 
computed in the FY76 BPF. In addition to being conceptually 
desirable, the arithmetic of the recommended procedure 
requires this and precludes the direct comparison of total 
state performance against a state standard, as illustrated 
by examples on the following pages. If they so choose, a 
.state could use the results to allocate resources among 
areas within the state* X 

^a. Computing a state score for each measure of per for*- 
ti - mance 

This section describes the Computational procedures j 
illustrate<^ with an example from one state, using only two 
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quantitative measures. For each labor area, its performance 
for the year (or other budget period) would be compared 
with its standard for each of the perfoinnance ratios. A 
simple method for making the comparison is to divide actual 
performance by the standard to arrive at a percent of 
accomplishment. (A modification of this would be to 
compare actual with expected, plus or minus one standard 
error. By definition, 2/3 of the areas will have performance 
that is within one standard error of expected.) 

Assuming three performance^ indicators in each area, the 
measures of perfoinnance compared with standard can be written 
algebraically as follows; ^ 

Pm1= IP/MYi actual 
\ i IP/MYi expected 

Prt^^ IPAJj actual 
^ IP/Ui expected 

PM3?f OF/Empi actual 
, i . OF/Empi expected 
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The first performance measure, Pm|- is ratio of actual 
niimber of individuals placed per man-year. The subscript 
(i) indicates that this is a measure for a particular area 
within a state'. PM? compares actual individuals placed as 
a percent of the nuillber unemployed (IP/U) with expected, 
and EM? compares actual openings filled as a percent of 
the number erfiployed (OF/Emp) with expected. For each^^of 
the measures, average performance in a given labor area 
would result in a score of 100, while higher or. lower 
petformance would yield a higher or lower score. 

Table III-7 on the following page, illustirates the - 
calculation in a state reporting 7 SMSA's for two of thpse 
■performance measures for which data were available: IP/U 
at the top and OF/Emp at the bottom. For each area, col. (1) 
shows the actual performance. Col. (2) shows the expected 
and Col. (3) shows actual performance as a percent of 
expected. The range in Col. (3) for IP/U is from 37.5 
percent to 67.3 percent and for OF/Emp it is frpm 45.8 per- 
cent to 191.8 percent, with Only two parts of the state 
exceeding the standards 

To arrive at a statewide performance measure, each of the 
measures rrust be separately combined into a' weighted 
average to% the state, "me weights are based upon the 
denominators of tfie perfoonanoe measures, and the appropriate 
weight for each measure is the fraction of the state total 
that is in the labor area. For the productivity measure, 
the weight is the percent of state man-years in each area; 
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TABLE II 1-7' 



Example of How to Compute a Simmary Performance 
core for a State ' fot Two Performance Measures 



Performance Measure, z - IP/U 



(i)* 
Actual 



(2) 

Expected 



(3). 
A/E % 



(4, ^^^I3h^ (5, 

# Unemp % of State 

(lOOO's) Unempx 



Weighted 
Average 
(6) 

[Sum of. 

(3) X (5)1 



SMS A 1 


30.4961 


81.2493 


37.5340 


12 .0 


4.86 


• 1.82 


2 


26.7327 


35.0216 


76.3320 


33.6 


13.60 ' 


10. 38 


' - 3 


19.0474 


33.1501 


57.4580 


45.6 


18.46 


10.61 


4 


35.9696 


93.2057 


' "38.5916 


16.5 


6.68 


2.57 


5' 


29.7352 


«€6.1604 


44.9441 


15.9 


6.44 


2.89 


6 ' 


32.4197 


55.6644 


58.2414 


16.9 


6.8^ 


3.98 


7 


30.2968 


55.67,52 


54.4170 


11.9 


4.82 - 


, ,2»62 


Bal.' of 


43.0449 


61; 9 790 


69.4508 


94.6 


38. 30 


2 5.60 


State 














Total 


[33. 11561 


[55.54681 




^ 247.0 


100.00 


- 61.^9 



State 



SMSA 



Performance Measure 3 - OF/Etap 



State 

Total 
State 



(2) 



(3) 



Weights 
(-*> (5) 
S Empl % of state 





Actual ' 


Expected 


A/E % 
— 


, (lOOO's) 


Empl 


1 


1.3100 


2.2993 


56.9739 


265.2 


6.05 . 


2 


1. 3553 


1.8280 


74. 1411 


557.8 


12.73, 


3 


1.0733 


.5595 


191.8320 


885.3 


20.21 


4 


1.5754 


2.6981 


58.3892 


4 37.4 


,9. §8 


5 


1.0797 


2.3566 


45.8160 


342 .4 


7.82 


6 


2.0725 


2.2525 


92.0689 


2 69.1 


6.14 


7 


1.6603 


2.2127 


75.0350 


219.0 


5.0 


of 


2.6993 


1.9464 


139.6817 . 


1404.6 


32.06 



[1.7863] [1.8101J 



4380.8 



100.00 



Weighted 
AVe rage 

(Sura of 
(3) X (5)1 




114.93 



RJC 
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the othe^ two mea^u^res the weights are the percent of 
unemployment^nd of employment respectively as shown in 
Col. (5) of Table 111*7. The algebraic formula for. 
calculating Jihe stats level score for PM^ (^nfeasures IP/U) 
isi 

PM? = ' CM 2 Ua 




The sivi^script (j) indicates thalt this is for a givefi state. 
The formula shows that the st^te score (pm3) is the sum 
across all areas of the st^te of the area performance.^ 
scores (PM^j) multiplied by the number unCTiployed in the area 
as a percent of the state ' s tot^ljanen^loy^ent (U^jAJ :)*? 
The weighted perf'ormance measures foir ^acM.,.^ea completed by 
multiplying CoX. (3) by Ctjl. (S) are shp^Yxl^Col. (6) , and 
the weighted Average for the $tate (tjaesum of the areas) 
is shown at the bottom, for individuals placed as a percent 
of the number unemployed, tl^e weighted average actual,, 
performance compare^ with eicpected is 61^,49 percent;; for 
openings filled as a percent of employment, the score' is 
114.93 percent. ~ \ ' z ■ 

SoTije'have suggested that it' would be equally as good to 
calculate the score for, each measure, as the ratio of state 
performance to the weighted average of labor area standards. 
To illustrate the error, weighte(3n averages of actual per- 
formance and expected performance were calculated. The 
weighted average of ^ actual performance is , by_ definition , 
equal to the actual s^ate total performance and is printed 

:ets under Cols. (1) and (?) . For IP/U, the 
weighteSh^erage actual was 33.12 and the 'weighted ayerage 
expected W£i^55.55. The ratio of 33.12 to 55.55 is 59.62 
percent. Tnis is less than the true weighted average of 
61.49. Similarly, for OF/Emp, the ratio of 1.79 to 1.81 is 
98.90, less'than the 't^rue avera^ of 114.93. ^ 

This result occurs because the larger areas of the state 
tended -to have higheij ratios of aptual to eJ^pected per- 
formance than did the smaller areas. The only way to 
properly reflect this is to compute the state total^ for eaoh 
performance measure as the weighted average at the ratio of 
actual to expected in each area. 

b. Computing the statewide BPF summary spore 

The statewide summary^ score would be computed as an average - ', 
of the scores for ea^ph of the individual performance 
measures^ This wou£<a be a weighted average if certain items 
were cqnsidered more important than the others . For example , 
it 'might be reasqnabie to^gi've more weight to individuals 



placed than to the other measures. The formula might then 
read: Using only the twp meaStires for which data are 
available, and Assuming that one might decide to give 
€0% weight to -individuals placed and 40% weight to openings 
filled, the BPF •f oi^ula would read as follows: 

) 



BPF^ = 60% PM^ + 4C33fc PM^ 
J 3 j 



Using the values from the preceding table, the calculated 
result for the state in our example would be: 



BPFj = .6 (61.49) + ,4 (114.93) 
" ^36*894 + 45.972 
» 82.866 



*The result of this equation can be called the "EPF" score, 
because it is analagous to tlie policy-weighted score which 
was computed cpmpar^ng state performance against the national 
average in the EPF for FV75 and FY76* The weights of €0% 
and 40% are analagous to the budget weights used in those 
formulas, which of course had more than two performance 
measures. Howevdr, in the recommended method, the state's 
performance is compared with its own standard: the weighted 
sum of the performances of each labor area. I^f a state had 
average performance, it^ would receive a score of 100; in 
our example, state performance was below standard, and the 
score is ^2,866. 

Therefore, the score can be used in future funding allocation! 
in precisely the sajne manner as the EPF summary score was / 
used in the past, to calculate the state's share oft 
resources* For example, assume that 20 percent of the 
annual allocation would be adjusted for performance, 
ignoring hold-harmless provisions, etc., the formula can be 
expressed in the following way: 

MV 



This formula says that the man-year allocation for statej 
for next year (my ^ + l)^ equal to its share of current 
national man-yeoErs^ f-'^y i ) , adjusted for performance 

+ .2 BPFj) , multiplied by the total national man-years 
available for allocation ( my t + 1 ) * In other words, the 
formula adjusts eacK state's share of national man-years 
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based on per formance , with the state's totaj, dependence 
largely o^ the national total available"; ^e result is 
quite similar in end result tp previous BPF formulas. 

If tl^e example state had 1000 man-years in the current year 
( MY ^ out of a national total of 25,000 man-years ( my^) , 
then the next step would be to calculate the state's share 
for^ the next year: 



MY = 1000 I- . i 

i 25000 t .8 + .2 (.82866)1 ^^"^ 

" .04 { .96573] MY*^+^ 
= .038629 MY*^+^ 

The initial share (1000/25000 or .04) is reduced to 
96.57 percent or .0386, because per^pnanc^ was below 
standard. If the national number ofS(fean^years available for 
allocation (MY- ^*^) remained at 25, 00?, b^e state would 
get only 965.7 in the next year. . ^ 

Allocation of marginal or supplemen^tal resources and 
estimating ES output ^ \ ^ 

Frequently, DOL asks 0MB and/or Congress for additional 
resourceSf and is ^ked in turn what outputs can^ i?e 
expected from the additional resource sC The pt^diction 
model answers the question by estimating the additional 
output that would be expected from supplemental or jnarginal 
resources . 

Ak ^lloccLtion which gives each state arid l^or area some 
resources but which pu'ts the largest share in the arQas 
with^the greatest expected' performance can be computed h.y 
giving each labor area (via the state) a share of the 
su^lement which is equal to its current, share of resources, 
adjusted for performance: 

uyS s WY t ^ 

^fj D_ BPF. X MY^ 

MY ^ 

This eqtiation states^^at the supplemental man-years for a 
given state could be allocated by applying the unadjusted 
BPF score to the state's starting point man-^year then 
allocating the supplement according to the adjustec3 share. 
Using our example state, it started with 1000/25000 or 4 
percent of national man-years. 'Its BPF score was 82.87, 
which, multiplied times the 4 percent starting share, yields 
a supplemental sharfe jequal to 3.31 j/ercent. If the size of 
the national supplement was 1000 man-years, the example 
state would get 1^1 man-years. 
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The extent to which the underlying predictive model assists 
in predicting what outputs can be expected where depends" 
upon the complexity of the mpdel* In Section B* of this 
chapter; a generalized standard form equation was written 
that shows individuals placed per man^year (IP/MY) as a 
function of external and internal factors* As discussed^ at 
that point, if the actual equation used to compute expected 
performance is linear, then the average IP/MY is equal to 
the marginal* Expecteji output from supplemental resources 
can be estimated simg'ly by multiplying either the observed 
IP/MY or the calculated expected IP/MX by the size of the 
supplement": 

Marginal IP = expected X MY^, or actual X MY 
MY ' 

It, however, the equation contains nonlinear or interaction 
terms, then the equation for the marginal product must be 
derived and used to estimate expected output* The derivation 
process, while simple to ecpnometricians , is too complex 
to explain in this report* 

Xfsing output standards at the local office level 

a* Within the 150 major labor areas 

The output standards to be developed by the methods 
described in Section C* , are standards *for an entire labor 
area, and are dev^eloped taking into account the external 
social, economic, policy and law factors that constitute 
the relevant environment within which the ES renders its 
services. With the exception of possible adjustments for 
the location of offices in different parts of SMSA's that 
cross state lines, the recommended methods would not result 
in explicit standards for individual local offices" (except 
where a single local office serves the entire area)* The 
"rational^ for setting standards that apply to entire J^bor 
areas, rather than individual local office^, was stated 
under D* 2 above* 

There are two questions that might be addressed in relation 
to standards for individual local offices within a multi- 
office SMSA* First, is it necessary to develop local^ 
o*Tice output standards? Second, is it feasible to do so? 

The necessity for developing local office output standards 
is related to the degree of autonomy of management 
exercised fey the local office manager* The local officn 
traditionally has been the level at which staff and other 
resources are managed and utilized to providje services, 
and the local office manager has been a very crucial person 
in the ES m^inagement struc^ture * 
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HoweVer, in recent years two. trends have developed that 
were observed in the field work research on input .standards. 
The first trend derives from the organizational approaches, 
developed by ES agencies to utilize job banks. As a 
minimum, most areas have centralized order "taking, referral 
control and verification functions, which necessarily ^ 
created a strong interdependence among of f ices . In addition , 
many areas centralized employer services ^nd other support 
functions". In the extreme case,. on6 area has 10. to 12 
service loca^tiojis whic^i prpvide only intake, selection and 
job referral sei^yices, with" all other functions handled 
from a single central Ibcatidn. This trend has reduced 
the autonomy of the manager of each service ^location and 
has strengthened the role of the area or disfctict manager, 
who participates in the formulation and implementation 
of policies, E^rograms and procedures. 

A "second, related, trend^is toward experimentation with 
different constellations^of office Ipcations and functional 
specializations. The experimentation .is moving in many 
different dij^^ections relating to horizontal and vertical 
structures and the use of information technology. (See 
also Aller, et al, pp. 25 to 29.) This requires the 
involvement of the area manager ^^in the selection of 
alternate service locations, the definition of functions, 
and the monitoring and evaluata,t>n of the -results of each 
test. Again, the trend lea<is^ feow^ird a stronger central ^ 
manager ^nd weaker local office manager i. 
T 

The question of feasibility must be answered in the context 
of the complexity of different patterns of local office 
location, functions and structures and the stability""' of '~ 
such patterns. There are several types of local office 
specialization. As noted above, there is a trend toward 
experimentation with offiPes having a limited niuuber of 
service functions. Traditionally , offices have speci-alized 
in service to specific occupational and/or industrial groups. 
The setting of local office output standards would require 
an evaluation of the effectiveness of each type of specifi- 
cation in different environmenPts , then the translation of 
the results into standards, taking into account the causal 
chains implied in tt^e model of the ES presented in Chapter II 
of this volume. As stated in Chapter II, considerable addi- 
tional research and development is required to complete that 
model, and. the volatility of the patterns would have to be 
reflected - ^ 

Our conclusion is that it "is not feasible in the short 
run nor is it necessary to develop specific output 
standards for specific Icffcal offices in a multi-office 
labor area. Output standards for the labor area as a 
whole can be used by area management as one input in 
evaluating the performance of individual offices within 
the area. Failure of an individual office to aoh'ieve 
performance equal to or better than the standard f<Sr the 
area, or success of an office in, exceeding the standard, 
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cannot be taken in 'isolation from other considerations 
relating to the location, structure and functions of the 
office in evaluating the overall success of a given office 
in meeting the objectives defined for it as one unit /in 
the local ES service network* / 

b. Balance-of "Statfe areas. 

Specific output standards adjusted for external factors - 
cannot be set for most individual labor areas, due to l^ck 
of suitable data on labor force and unemployment for SMSA's 
other than the -150 major labor areas and lack of publis^ied 
data on employment for non^SMSA labor areas. Until the data 
gaps can be filled, we recommend setj:ing standards for 
balance-of-state areas^ which include all labtfr areas outside 
the 150 major areas. Each state would have a. single set of 
standards, adjusted for state-level external factors (net . 
of employment, unemployment, etc., for any^of the 150 areas 
withiTi the state.) While usable for funding alloCatibn' 
purposes, the published standards would not apply directly 
^o any labor area in the balance-of-state class. State ES 
Agencies, however, 9an use internal estimates of the factors 
in conjunction with the estimating equation or typology 
to calculate output standards for individual areas. While 
not as sound as the published standards, these could be 
used as valid, standards for internal evaluation purposes and 
perhaps as an input to planning^^he intra-state allocation 
of resources, * ^ 

A dilemma in uging output standards to allocate resources 

l€*.wa^ implied- in preceding sections that, following the 
philosophy applied in the BPF for previous years ^ a state 
whose overall, performance was equ^l to the standard for 
the state should receive the same share of national ES 
resources in the next year as it had in the previous year 
(this ignores, of course, "productivity increases", - 
definitional changes, and other factors that affect the 
total amount of resources available for allocation 
through the performance-based budgeting process.) . - 

It is' clear from examining the research results that, f 
when alL external economic ^^nd social, policy and law 
factors ar^ taken into account, there would be quite a 
wide range 4n expected performance among the states , even 
if ^11 states .and areas had performance equal to their 
standard, because the standards will vary dramatically. 
For examplfe, for F¥^4 at the state level, r expected pro- 
ductivity estimates ranged from 89 to 206, with 4 states 
having estimates under 100 and 4 states haying estimates over 
190,* nearly twice as high. At the^ labor area level, expected 
values for IP/U range from less than 10 to over 150 ! I 

The wide range in the estimated standards computed to date 
with the incomplete model developed in this project is 
attributable to the ^wide range of economic, policy and other 
factors among the areas* Further refinements in th^e model 
may lead to even wide^ ranges in the standards, as more of 
the variation in .actual performance is, accounted for. ~^ 
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Therefore, if output standards are established for use in 
the funding allocation process, similar results can be 
expected, although the specific methods used will have a 
bearing -on the resultant ranges of performance standards. 

In theory, the purpose of -reflecting the external factors 
in setting putput standards which will be used in the 
resource allocation process is to adjust for factors beyond 
the contrtJl of ES management and to come up with standards 
against whicli ES management can be judged. States whose 
performance exceeds the standard are assumed to have good 
management and are to be rewarded accordingly, while states 
whose performance falls below, the standard are assumed to 
have pQor management. 

The interpretation of the research results is 'that certain ^ 
states having average management capability ■ can reasonably 
be expected to have performance levels at least twice as 
high as certain othdr states also having average management 
capability. The credibility of the research results have been 
challenged, because on-site evaluations have identified numerous 
examples of poor management practices in certain areas where 
the statistical analysis results indicate that actual per- 
formance is equal to or better than the (low) standard; the 
converse is also true. One hypothesis (not easily testable) 
is that the environment in certain areas is so detrimental 
to ES performance as to produce organizational frustration, 
manifested as poor management, low morale, etc. 

This leaves th^ policy maker with something of a dilemma; 

On the one hand, it is desirable to reward good manage-- 
■ ment, implying that the absolute level of performance 
^ is not being judged, only actual performance relative 
to a reasonable standard* ThiTS approach is advantageous, 
particularly when in the past it has been the l^rge 
northern and eastern states who have suffered most 
through EPF. If the standards are lower for such states, 
it is less likely that they will suffer cuts in the 
future. 

On the other hand, the ES exists to serve job seeker:s 
and employers^who request its services. Irrespective 
of the quality of managements or the stability of th^ 
service organization, it seems reasonable to allocate 
the resources to states where the greatest number of 
clients can be served. It is often argued that resource 
reallocations through the BPF have in some sense 
penalized the people living in the states Ipsing 
resources - 

The converse of this argument is that leaving resources 
in states where actual of expected productivity is low 
{irrespective of quality\of management) penalizes the 

• ■ ■ 
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residents of states where productivity is high* For 
example, a state whose productivity is 200 individuals 
placed per man-year can serve twice as many people per 
unit of resource than a state whose productivity is 
only 100, Job seekers in the first state lose in the 
ratio of 2 to *1 when (marginal) resources are re- 
allocated to the second state. 

The policy maker must address this dilemma, taking into 
account borh the political and the program implications 
of different solutions, prior to decidinc how the adjusted 
output standards should enter into the funding allocation 
process,\if at all* 
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Chapter iv 
iJETHObs FOR ESTABLISHING INPUt STANDARDS 



A. Introd\j£tion! Practical Definitions .and Uses of Input 

Input standards are models of the placement and placement support 
processes which can help diagnose the cause o-f poor performance * 
when actual operation^ are compaored to them. Their purpose is to 
help improve the Employment Service, particularly in local 
placement operations. The development of these standards is not 
a simple matter and the"d:^f f iculties of the task were recognized 
from the beginning of the project. In fact, doubts have been 
expressed that standards could be established without first 
answering long range questions on the specific missions, purposes, 
objectives and role6 of the ES such as, *'is the Employment 
Service primarily/ serving efnployers or serving applicajits?'* or, 
"Does the Employment 'Service aim primarily to maximize placements 
or to reduce the unemployment rate?" When this study was started 
these questions already had a long history; pethaps they will 
have as long a future. It was clear then that the ES could not 
wait for them to be answered; the practical needs for an effective 
tool to carry out the purpose of improving the placement process 
was too urgent. An effective response to the current demands for 
an effective and efficient Employment Service requires that 
adequate mana<gi|^ment tools ^ put into the hands of state and local 
managers as rapidly as possible. 

An Integrated Management System that combines the output performance 
evaluation components of the Balanced Placement Formula with 
diagnostic and prescriptive analysis can be developed* without 
waiting for the answers to long range questions. Studies of the 
cost effectiveness of the placement process may take years; now 
there is an urgent need to make' the current placement process as 
efficient as possible, at" least until alternatives are developed, 
tested, and implemented. We therefore began our study by 
proposing a model for the placement process, that depicted the 
current functional activities and the relationships between them. 

An Overview model is shown in Figure iv^l on fehe following 
page. Resources allocated to each functional activity are inputs 
to the placement process. The. inputs are assessed by comparing 
the "actual performance levels with standard levels to be determined 
through methods described in this report. The efficiency of each 
functional activity is assessed by comparing the resources 
required for the measured\performance level against the standard 
model resources. In thi^project we have performed the following 
tasks; .. . > 

/A 

o Sel(ection of the measures of performance. 
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• Development of a method to measure the levels of 
performance for each functional activity* 



• Development Of a method tQ determine the resource 
requirements to meet those performance levels when 
the functional activities operate efficiently* 

Our project w^s designed to , set the stage for a larger survey 
tjiat would establish the actual performance standards* Our 
objective was to develop methods which would* be used in that 
survey* After developing the methods we tried them out in 
several loc^ations to demonstrate their practicality and 
capability to develop the standards* ^ In these pilot trials 
we developed a self-application procedure for conducting the 
surveys that will significantly improve the scope of the next * 
step by reducing the need for a member of the study team at 
every site on every day of the -survey* A Handbook for Analyzing 
Local ES Performance (Voltime 4 of this report) has been 
developed for collecting time utilization data at the local level; 
for computing efficiency measures, service percentages, and key 
quality factors; and for collecting the data needed to establish 
and validate input standards* In addition, the Handbook can be 
used immediately for local office analysis* 

To illustrate the form in which the data can be analyzed and some 
potential uses of the data. Table IV-1 on the foMowing page shows 
the results' of applying the methodology to tw«?"S*Q|cal offices^ 
Part A of the table shows the percent of resources allocated to 
each category in the placement process* Ttie categories were 
defined so that they can be related directly to codes used in 
the official time distribution system without being limited to 
those codes, (The alphabetic codes were developed and used in 
the data collection at the local office level to avoid confusion 
between time recording on the special survey time ladders and 
time coding on ^the time distribution sheets*) In addition to the 
items shown, tfie. Handbook* also contains methods for measured 
inter-activity ratios, such as referral-to-placement ratios, 
individuals counseled or tested as a perceni of intake, etc* 

ES- input standards, when established, will prpvide a model, a 
measure and a comparison upon which ES management can build - 
performance improvement plans* A primary use of the standards 
will be as a tool in diagnosing the causes of poor performance 
in labor areas that fail to mee't output standards, not becaiise 
these are, the,, only areas that could achieve higher output 
performance, but because the performance-based budgeting system 
tends to give an incentive to concentrate on s\ich areas* To use 
the standards, it is first necessary to perform the resource 
utilization measurements described in Volume 4 in the labor 
Arc.-\{G) p.nd local office(n) of interest and to calculate the 
Several statistics that characterise input utilization* The 
results can then be compared against the appropriate standards * 
for that type of labor ^ea to identify differences which might 
indicate ca-uses of poor performance* 
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Table IV-1. Example of Data Compilation from On-site Survey 
of 2 Local Offices 



A. % Resource Allocation for Placement Process Component Activities 



Placement and Support Activities 


L0#-1 (%) 


L0#-2, (%) 


AT 


APPLICATION TAKING 


8. 1 


14. 5 


CS 


COMMUNITY SERVICE 


4.D 


' 0 .6 


CO 


COUNSELLING 


9 ^ 


11 . 0 


ES 


EMPLOYER SERVICE 


6.1 


2.2 


FC 


FILE SEARCH /CALL- IN ' 


77T 


TTET 


JD 


JOB DEi^ELOPMENT 


TTT 




JIS 


JOB INFORMATION SERVICE 


1.5 


2.1 


JO 


JOB ORDER TAKING- 


1.9 . 


2.7 


RE 


RE CEP DION 


4.5 


6.6 


RC 


REFERRAL CONTROL 


4.3 


0.3 


RI 


REFERRAL INTERVIEWING 


9:8 


11.3 


TE 


TESTING '/ ■ 


2.5 


0.1 


VE 


VE RI F I C AT I ON / VAL I D AT I pN 


2.0 


1'.3 




Subtotal 


5^ 


58.3 



Non-Placement Activities 



XC CETA 

XE EFS STUDY 

XF CLERICAL 

XI STCK • ' 

3CJ JOB CORPS, FOOD STAMP, UI , ETC.. 

XM MEETING, TRAINING 

XS SUPER., MANAGEMENT, ETC. 

XV VACATIo'ri, ANNUAL LEAVE 

.XX PERSONAL, COFFEE BREAK, ALL ELSE 
Subtotal 



Total 



0.0 


, 1.6 


0.5 


1.0 


11.2 


8.9 


4.4 


2.4 


1.0 


2.7 


6.6 




6.5 


^4.4 


5.8 


r 5.0 , 


7.2 


15.3 


?J72 


41.7 


100.0 


100.0 



B. Selected Service Ratios (percent of Traffic) 



Application Taking 
Referral Interviews 


44.0% 
60.5% 


4 
5 


;.9% 

179% 


C. Selected Quality Factors:, 


r 



% Responding Employers Answering YES 



Does an ES representative contact you 
regularly? ^ . 75% 

When an ES representative visits you, does 
he/she appear to be knowledgeable of the 
current labor market? " 93% 

^ i. • : 
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Monthly Placements per Equivalent Position 

64 . . 



45.8 



43% 



50% 



28.6 



Pending the establishment of input standards, the results of the 
measurement can be utilized by comparing the statistics with 
existing sources of related BLnformation* The local office manager 
has planning and budgeting standards of conparrson for resource 
utilization and servlr^ze ratios from the Plan q£ Service System and 
has data on performance per unit of resources^rom POSARS* 
Comparing the results ot the survey with the data from official 
sources, the Toanager can compare his actual resource util"lzation 
and intermediate outputs with his plan, and can validate 'thfe ^ 
official figures* |He can also review the data on key quafii^ 
factors for each a<ptivity against available (publisjied or personal) 
standards of performance and identify staff training needs in the 
state-of-the-art ♦ - ^ 

* . -r 

f ^ 

The, area manager or field supervisor, given data frotn several ^bcal 
offices,- can compare input resource utilization with intermediate 
and final outputs to identify potential areas for correction* In 
Table IV-1, for example, LO 2 has a much lower number of monthly 
placements per equivalent staff position than does LO 1* LO 1 
devotes relatively 'mojp^' resources to employer service and to file 
search 4nd call-ini ^nd^iess to applipation-taking and counselling* 
has highOir service ratios for applications-taking and coxinselling* It has 
higher service ratios for referral interviewing, and a better rating 
on employer service quality -factors* A field supervisor could use 
such comparative ;da,ta , even in the absence of validated standards, 
to examine ^he causes of such differences , inc luding the effects 
of external e\;onomic factors, etc*f and to develop a plan for 
improving performartce in the second local office ^ 

This example illustrates the potential for Applying such methods in 
the diagnostic process evert on an. interim basis* It also illustrates 
the potential pitfalls of applying validated- standards* Input 
standards can be "developed in accordance with the methodology and 
plans^ which standartis can never be used in a rote cookbook -manner 
to presdribe changes in the orga'nization or functioning Vf the ES* 
TJier need for such changes must always be identified on tM basis 
^ of sub-standard output performance, ^nd corrective plans must always 
take into account the specific realities and 'constraints of the 
individual labor area* The Handbook for Analyzing^ Local ES 
Performance , (Volume 4 o-f this report) describes in detail how raw 
data can be collected, how the data can be compiled into standard 
forms, how observed patterns can be' compared to the standards, and 
how the comparison can be used in developing performance improvement 
plans* 

The ideal objective of the next major phase of Input standards 
development would be to establish and validiSte patterns of 
resource use , service ratios , and ef f iciency and- key quality 
factors that discriminate between areas living h^h levels of 
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output performance, (under given", sets of external conditions) 
and all other areas* * However, the current state-of '-the^art 
does not permit the "validation)' of input standards* Valida* 
tion of standards requires the analysis of input patterns in 
relationship to output performance, and the iden^tif ication 
of input patterns th^at discriminate between high perfomers 
and low perfoinnners , hold;Lng all' ottier factors constant* 

The use of output standards and the comparison of actual 
performance with the^ standards pemits identification of hig)i 
and low performing l^^or areas^ holding constant the effects 
: of exter-nal economic and policy factors* To validate input - 
standards additional factors must be held constant: local 
office location, overall ipanagement quaplity, staff skill level 
and morale, and quite possibly the mix of applicants and jcito 
openings*. This would pemit the observed patterns of inputs 
to b^^-4;est:ed, net of all other influences, to identify those 
patterns^^Millich discriminate between highland low performers* 
While techniques hav^ been developed for measuring location 
and management quality, they have not been sufficiently tested, 
and integrated into a comprehensive system to pemit -US to 
factor them into an analysis in the immediate future* '^ 

Therefore, an achievable short**run objective is to identify 
patterns of input utilization characteristic of high performing 
labor areas and of offices within thosjf area^, ignoring the 
other internal factors and deferring the question of validation 
to a future t^m^ wheii the ,-state-of -the^a^t permits validation ; 
The steps needed to achieve the objective include measuring 
input utilization in a sample of laJbor areas^ identifying input 
^atterns> and preparing input guidelines for different types ■ 
of labo£ areas* The steps outlined in this chapter are ^ " 

designeg:5to finalize these patterns* ^The recommended approach 
will result in the setting- of stan/iards "for each of the 
13 functions identified in Table ^-1, biased on the average 
level for each measure in high performing offices in each ' 
type of labor area* It is quite (possible *that when it becomes"^ ' 
possible to validate input, it will be found that not all of the. 
functions nor all three measures of each function, discriminate 
between high performing areas and other are^S * CpCt is further 
possible that periodi,c revalidation*' of the s£anaards will show 
that different functions or measureis are va3^id, as ES service 
obje<^tives and methods gh^inge^over time* i ^ 

The current statyas of the- program for development of ES inpOt 
standards is ^t hat methods have be^n developed and tested: for 
"measuring the utilisation of ES staff time iii the local 
office an pei?formaKce of 13 placement an^ placement support 
functions and ^ non-placement (os/erfiead) is;^ategorres , measuring 
activities and'* services resulting from plair^ent functions i 
'and measurlJig keyjquality factpr^ as^ocAat^d with' those 
functions; for otTmputing ratios, mi^^^s per unit of service, and" 
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resource distributions ; and for aggregating results to the ^ 
labor area level, and, where feasible, to the state lev^l. 
Section B, of tills chapter presents plans for classifying 
labor ateas by type and for selecting* samples of labor areas 
and local" of f ices for different dat^ collection purposes^ 
data analysis problems, and implications for validating and " 
revalidating standards. Section C, discusses an additional 
use of^ the data. ^ " 

Methods of Data Collection and Analysis 

The measurement of ES staff time utilization' and service ^ 
patterns for all of the approximately 25,005^to 30,000 ES 
staff members in all 2500 to ^000 service locations is- neither 
necessary nor desirable for collecting the data Heeded to 
establish input standards. The problem is to design an 
efficient, practical approach to selecting a sample for the 
measur'emen*t, the results of which can be used to establish 
input patterns characteristic, of - high^ pe^rforming labor areas 
of differ&nt types. The recommended design is*a stratified 
sair^le of latoor areas resulTiing -in a sample of \^bout lo' 
percent of E& local offices and staff. 

1., The sample of labor areas * ' 

As stated previously, the labor area is the preferred 
unit of anS^sis, because ^t is an integrated economic 
and social System within wSlich the ES provides services. ' 
For practical purposes, the relevant size of the universe 
is the ISOjSMSA'is that are defined as major labor areas 
plus 48 toalance-of-state areas. 

The first step in tjie sample design is'*to stratify labor 
areas^by external Economic conditions found to be 
significantly related to ES output per forfnance . The 
\ factors identified in previous research are size, the 
stable ones of industry composition, and *labor force 
composition; and the more volatile ones*of unemployment 
and growth. The actual stratification or grouping of 
' laboiN areas by type "can be done using these variables 
(or others found significant in further development of 
output standards) and the combination of factor analysis 
and hierarchical clustering as described in Chapter HI.C.. 
l.b. The result would be groups o^, areas as similar as 
possible to each other witt^n each group. Therefore, the 
groupings would be used not only for sampling but also for 
the typing pf areas in setting standards. For. the purpose 
-^ pf using the input standards, the list of the labor arran 
in each type must be published along with the standards for 
each type. 

Using th^ comparison of actual perfarma;ice versus outsput 
standards, labor areas can further be stratified into 
- high, medixjm^ and . low performance groups. If the concept 
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of a standard error or standard deviation is used as the 
criterion, then within each stratum, 1/6 of the areas 
would be high performers, 2/3 would be medium performers, 
^and 1/6 would be low performers* Across all strata there 
would be about 33 areas in the first class, 132 in the 
second, ahd 33 in the last* - ^ / 

The next step is to use the stratification to seldtit an 
actual sample of areas* If it were feasible to both 
establish and validate input standards, then -either of 
two sair^ling approaches could be used* One would be to 
select a sample representative of the entire universe; 
multi-variate statistical analysis techniques would be 
used to identify the input elements 'correlated with 
higher performance* 'The other would be to select only 
high performing and low performing areas within each 
type; analysis of variance could be used to identify the 
input measures that discriminate between high and low 
performing areas* 

Both approaches would encoXinter difficulties in a sample 
of 50 areas, because the ntmiber of observations in each 
class of area it^ight be too small to permit use of the 
statistical techniques, requiring the combining of strata 
or the U£fjB of only a few strati to start with* 

GjLven that it is not possible at the present ti^me to 
validate the input standards, one reasonable recommendation 
is that the sample be limited to the 33 high performing 
areas, using the data" within each type to calculate averages 
for each of^he inpUt measures; Even this^^approach maj/ 
encounter problems of a practical nature. Research completed 
to d^ite using data from 80* SMSAs and 48 balance of state 
areas results in only one of the 12 largest SMSAs falling 
into ^he high performance group* None of the. three largest, 
containing about 120 or 5 percent of the n'Stion's ES 
offices and possibly as much as 10 percent of the nation's 
ES staff, would be in the'sample based on available resultsV 
Since all three are different from each other when 
measured by both internal and external factors (except 
size) , since they seem'to be different from other ^ireas, 
.and since they consume such a large share pf ES resources, 
it seems highly desirable that they h4 included in the 
"Sample * ^ , . 

i \ I / * 

A feasible solution to the problem^^^ to divide the 
imiverse into two pa;i:ts: SMSi?^over 1 million in 
population^, and major labor are^^ tinder 1 million in ^ . 
population plus all balance of state ^areas* The 12" 
largest SMSAs are. estimated to contain approximately 210 
local offices and 3S00 staff, or aBout 15. percent of the 
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national staff total* Data collecttLon within such areas, 
while important to the total national picttire, requires 
further stratified sampling* It iTg* quite possible that 
the prior factors in the causal chain, particularly the 
location of offices, would have a dominant, effect on ES 
performance in large SMSAs with comp^l^x transportation; 
taking this effect into account, it would be necessary to ^ 
stratify local offices by functional specialization and 
by occupational/industrial specialization* It might also 
be necessary til sample work units within local offices, 
stratifying by functional unit and^ even by job description* 
The data are not currently available to begin to propose 
a plan for such sampling, and i^^ is recoramended that 
measurement of input patterns in. these areas be deferred 
to a later phase in the development of input standards* 

The recommended sample, then, reduces to selection of the 
high perfonaing labor areas from among the remaining 
138 major labor areas, of which there would be 23, plus 
the high performing balance-of-state areas, of which 
there would be about 8* It -is estimated that the 2 3 SMSA's 
should average between 5 and 10 local of f ices , while 
balance-of-state areas should average about 30 local offices* 
If the 23 SMSA's aVeraged 7 offices, all could be included 
in the survey for a. total of 161, leaving .39 officd^s to 
be selected from the 8 balance-of^state areas* The highest 
5 performing offices per state could be selected based on 
state data* Using this design,' the balance of state areas 
woiild be a single stratum with 8 members* The 39 offices 
selected would constitute a J.€*8 percent^ samp-le* 

The recommended method would result in a sample of 200 
offices containing an estimated 2000 staff, about 9*4 
percent of the staff outside the 12 largest cLties* . 
Returning to the question of the appropriate number of 
strata^^ excluding the classification by output performance 
in relation^-tp standard, it appears that a desirable 
number woujtd bk 7 classes of areas, allowing an average of 
5 or € areks per class of SMSA plus 8 areas in the balance- 
of-state " stratum! which would permit a computation of 
the variatiwr^mong areas within each class to accompany 
the confuted average for each performance measure for ^ 
each type of are^* 

The sample of time 

The /data collection design presented in' Volume 4 discusses 
the choice of at-, one-week period in which to conduct the 
survey in each local office. The period of data collection 
is limited to a one-week, ^efiod to minimize the burden^ and 
it is recommended that the survey be done in a continuous 



five-day period to .maximize training effectiveness* 
The one*week constraint raises the possibility that 
sample results will be unduly affected by seasonal 
factors, by unusual local conditions and by the 
sainpling error. Each of these aspects needs further 
consideration* 

It is quite likely t^iat seasonal factors will influence 
the results of the data collection* If collected in 
the Fall staff time utilization patterns and service 
ratios may. reflect a general seasonal decline in hiring 
activity, while data collected in the early Spring could 
show the opposite effects* Data collected near the end 
of the school year would show the seasonal, effects of 
large numbers of youth applying for spminer jobs* Data 
collected in the Summer might show the effects of temporary 
ES "staff . reassignments and absences to cope with the 
traditional vacation period* Holidays occurring within 
a survey we,ek might shift worfe loads or leave factors* 
It is strongly recommended that a survey week be chosen 
in conjunction with state ES management that does not 
contain a holiday* Beyond this, the only knov/n method 
for adjusting for seasonal effects is to collect data 
during each season (perhaps during the mid-^month of each 
calendar -quarter) and compute an annual average* 

The extent to which unusual local conditions during survey 
week may affect results is less clear. Obviously, if a^ 
majot training program, management review, or other 
controllable event is scheduled, the survey would yield a 
higher than normal percentage of nonplacement time, and 
it is recommended that the data collection be, scheduled 
away from s-uch events. -It is also possible that major 
changes in economic conditions, such as a massive layoff, 
could affect performance in a given area. It seems likely, 
however, that the effects of such events related to the 
business cycle would average-.out over a number of similar 
labor areas* The only feasible adjustment method would be 
to sample at different points in the business cycle* This 
could be combined with the quarterly measurement plan dis- 
cussed above. 

The sampling error associated with the survey Week can be 
computed. Since thp national average leave factor is 
about 15 perceht, the average number of weeks worked by 
ES staff nationally is about 44*2* One week, therefore, 
is a 2.26 percent sample of each staff member's time on 
the average, if leave taken during the week is excluded 
from the calculations; If time is sampled quarterly, this 
results in a 9 percent sample of time* Sampling error. 
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which assumes an unbiased estimate of the true value, is 
a function of the square root of three factors. It 
increases with the measured variance. It decreases with 
an increase in the sampling fraction, but the relative 
increase in sampling error for percentages is only 3 
percent for an increase from a sampling^fraction of 2 
percent ^ a sampling fraction of 9 percent. More impor- 
tantly, the sampling error decreases wi^ an increase in 
the sample size. For the current problem, this is impor- 
tant only with respect to the balance-of- state estimates, 
where the number of offices per state is small- However, 
since the balance of state is treated afi a single area in 
each state for which the estimates are to be computed, the 
relevant number is the expected number 9f staff (estimated 
at 40 per state) rather than the number of local offices. 

Computation of input standards by type of labor area 

The data collected ^from the sample of labor areas will 
be used to compile the four types of input measures for 
each of the 13 functional activi-ties and nonplacement 
time;, for each of the 31 sample areas, using the detailed 
methods contained in the Handbook for Analyzing Local ES 
Performance^ The result will be 31 sets of input measures. 

. / . - * 

J^e data for the areas would be grouped by type of axea - 
5 or 6 SMSA types and I balance-of-state type. To calculate 

^ther irlt^ut standard for each type of labor area requires 
.two related computations. The first step is to calculate 
the average across areas within each type for each of the 
input measurjes, minutes per unit,* service percents; percent 
of resources, and key quality factors. (Of these, only 
the percent of resources is relevant to nonplacement time-) 
The .average for each measure Is the initial ^nput standard 
for each 5S function. -The second step is ta compute the 
standard deviation for each of the measures. The standard ' 
deviation will be published along with the . average to 
indicate the extent of variation observed. 

Table lV-2 on the following page sHows an example of how 
the resultant input standards would Idok when published 
for one type of labor area. In using the standards, ES 
management would look up the particular leU^or area under 
review in the list of labor areas by type, then consult 
the tables of standards for that type of labor area. 
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Table lv-2. Exait?>le of Quantitative Tnpirt Standards 



Activity 


Minutes Per 

W-l i ^ ^ y'ltJ.O X X ^ Y 


Percent of 

Trt J* 1 R&^ rtiil* 


Service 


Measures of - 


Average 

iftP R^csot ion * Rana^** 


3.0 
4 ^ 0 


4.5 


5.0 
1 n 

J. * w • 




Average 

AT AppXicat ion TaKin^ ^an<^6 * 


11.5 
1 5 


e.2 


44 
10 


0.025 
- 010 


Average 

CO Counsel li na R;inae * 


36-3 
' 5.0 


2. * ^ 


42.5 


2 

0. 5 


Average 

TE Testina P^nnA* 

L A w ^ ^4 1 ^ 4- ^-£14 1 ^ 


12. 1 
1.0 


2-5 ^ 
0 . 5 


7-0 


0-007/0-15 /0-050 
0 002f 0 03/ 0 005 


**• Ave rage 
RT 1 T ntervi ei wncr R/^no^ * 

1 C It^^JIrU^ L^-ili^Q-k > ^ Q 


10-1 
2-0 


9.8 
1.0 


60 " 
15 


0 05/ 0 5 


Average 


5.3 

±. M A. 


7.1 


56.0^ ■ 
20 




Average 

..TT Job TT\"f oi-T^A 1 1 OT\ S**i^vl r*d R;5tlaf>* 


4-e 

^ 0.5 


1-5 
0 7 


5 




Average 


.13-4 


1-9 
n 7 


1.0 

A 1 

\j * J. 


s 


n V ^ 

RP R** Ffil^l^a 1 Confr l^nl RaflCT*^* 


4-3 ^ 
0-5 


4.3 
0. 5 


J. Qw 

40 




Average 

W Verification/Validation Rang^ 


1-7 

0.2 " 


2.0 

0.2 


385 , 
70 




Aye rage 

ES Employer Service Range* 


103^1 
30-0 


6.2 
1.0 


30 
5 


^^^^^ rife 


Average ^ ' 

CS Community Service Range*^ 


NA 


^ 4.0 
/ 2.0 


' NA 




Averag^ 


5-9 
1.0 


2.7 

1.0 I 


50',0 


Q*2 / 0*50 


JD Job Development ^^—^ Range* 


•10. 


0-05 / 0,05 



^Range is equal td the average plus or minus one standard deviation. 
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Aggregating and Disaggregating Input Standards for Use 
at the Local Office i State and National Levels 
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As in the development of output standards , the methods for 
developing input standards lead to standards characteristic 
of labor arpas, more specifically the 150 major labor areas 
for which data on external economic and policy factors is 
available plios balance-of- state areas* For reasons discussed 
in more detail in the previous chapter, wje believe that both 
output and input standards should be set and used primarily 
at the labor area level, because this is the level at which 
ES resources are applied in service to an integrated economic 
and social system, although for practical reasons it is not 
possible to do so for individual ^areas outside the 150 major 
labor areas , at the present time ♦ 

There is, however, considerable interest in the use of 
performance standards in local offices which requires dis- 
aggregation N^low the labor area level, and in state and 
national offiwes, whicl^ requires aggregation above' the labof 
area level* The subje<ijt was< discussed with respect to output 
standards in the previoiis chapter, wherein a weighted averaging ' 
methodology was des^ibed^ for using output standards at the 
state level ♦ 

Fpr using input standards at the local office l,evel., one class 

of the measures can be applied directly* Thes6 are ttie key 

quality factors associated with each activity* The field 

work for this project leads u^ to believe that the specific 

measures of quality are generally applicable to any office, ^ 

because they represent significant dimensions of the accepted 

state --of- the --art for performing ES services to applicants and 

employers* The data collected for the purpose of establishing 

input standards should be analyzed to test for variations 

among areas* 

r 

The use of standards for minutes per unit of service , service 
percents , and percent of resources will vary depending upon the 
type and degree of specialization of x^ffices within an area* If 
offices are not specialized in any way, the standards may be 
applied directly* If offices are specialized along occupatronal/ 
industrial lines, the mix of applicants will vary among offices, 
and the minutes per unit and the service percents will also vary* 
The methods developed to date would not permit derivation of a 
standard for each local office* If offices are specialized 
along functional lines, an estimate of the percent of resources 
can be derived from the standard for the area by assuming a 
constant nonplacement percent and reallocating the standard 
percents among functionsperformed in the office under study* 
For example , consider an office whose functions are limited 
to reception , application taking," referral interviewing, file 
search and call-in,* and job development, with all other functions 
performed centrally within the area* As shown in Table IV-3, 
if the share of nonplacement tiine remained constant , the place- 
ment time would be allocated among the 5 functions'* 
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Table IV-3. Calculation of Percent of Resources 
fds^a Limited Service Office 



RE 
AT 
CO 
TE 

RI 
FC 
JI 
OT 
RC 
W 
JD 
ES 
CS 
X 



Labor Area 
Percent of Resources- 
(1) 



4.5 
8.2 
2.1 
2.5 
4.5 
7.1 
' 1.5 
' 1.9 
4.3 

Si 2.7 
6.2 
4.0 
43.2 



\ 



Loc^l Office 
Functions 
(2) 

4.5 

8.2 • 



. 4.5 
1.5- 

2.7 
4>.2 



Percent of Resource 
Allocation by Function** 
_(3) 

11.9 
21.8 



11.9 
4.0 

7.2 
43.2 



Subtotal Percent of 

Non-placement Time 



43.2 



43.2 



Time to be Allocated^ 



56.8 



I 

^Computed by subtracting nonplacement^tX) |>ercent from 100. 
**CQmpuVe by: 

(1) Add 'up the standard placement percents in coX* (2). 
Total is 21.4 

(2) Divide the total by the share of time to be allocated,, ^ 
56.8. 21. 4/56.8=. 3768 

(3) Divide each standard placement .percent by the result 

' (.3768) to obtain the allocation for the local office. 
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Aggregation of input standards to state or national totals - 
encounters similar problems. Key guality factors probably 
can be used without aggregation. Certain of the minutes* 
per-unit measures can bo r,ggre^ated through a weighted sum- 
mation process, where the weights are the share of the state's 
Service units in each type of labor area,. For example, the 
minutes per unit for counselling can be aggregated by multi- 
plying the standard for each type of labor area by the percent 
of applicants in that/ type of ^abor area, then summing to a 
state total^^.. For measures where the denominator is not a 
statistic or service (reported to ESARS, such as call-in attempts, 
the aggregation can Ohly be performed if the activity has been 
measured in all loca\fd offices and areas. A similar problem 
pertains to the service percents. A state standard for percent 
of resources can be computed by weighting the ^andard for 
each type of labor area by the share of state man^years in 
that type of- labor area, then summing to a state total. The 
logic of t^ese calculations can also be applied to the computa* 
tion of national standards. ^ - * 

An important question with respect to the computation of inputs 
standards at the state or national level is the interpretation 
and use of the results. Input' standards can serve the manage- 
ment of the Employment Service as a diagnostic tool and an 
aid in planning for improvement. in our concept of an integrated 
management system, the diagnostic tools (input standards and 
others) are ca;Lled into play primarily when there is a measure- ' 
ment* indicating that the labor area has achieved poor performance 
In other special cases / state management might wish to conduct 
a Performance Stajidard Survey in ^a large city in the state 
even if that labor area had good performance because of'the' 
importance a large labor area has in the state as a whole. 
Management might reason that even a small improvement On ^ 
good operation in a large area would be more beneficial 'than 
a large improvement th^y might achieve in the smaller areas 
in the state. ^The important point about all of this is that 
input standards are utilized after the output standards have 
indicated where the high and 16w per former s ^re . They assist 
management in reaching the potential of each labor area in the 
state' and they are naver used without an analysis of the output 
performance first . Although it may be desirable to estimate 
the optimum percent of applicants who should be counselled or 
the optimum share of '^resources to be devoted to a particular 
activity, the current stage of development only permits U3 to 
identify what high performexs have actually done. We can 
say "what is" as far as high performers are concerned; we are 
not yet able to say "wh^t should be". 
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A PL^ FOR DEVELOPING AN INTEGRATED MANAGEMENT SYSTEM 

The preceding chapters of this volume have detailed the cdncepts 
and methods developed to date for the establishment o£ output 
and input performance standards for the United States Employment 
Service. The purpose of this chapter ^is to define the specif i'c 
steps that can be taJcen to implement ES performance standards 
during the next 12 months, to present approaches , to further steps 
that would be needed in the future, and to summarise the conceptual 
and practical limitations of the performance standards that would 
restflt frpm the j^ecommended methodology, 

it 

A. Plan for Establishing ES Performance Standards 

There are three crucial milestones that should be met in the 
next ye^r in the development of ES input and output performance 
standards. The first of these is the beginning of the Fall 
hearings on the ES budget at 0MB, which start on or about 
October 1, but delays in completing the present report m^y 
preclude meeting that date. The pflan presents steps that would 
result in an initial set of ES output standards based on a 
static model that could be* us^d to project ES performance for 
FY 1977, .as ojae input ^to the budget process. The next milestone, 
is the date fo!r publishing BS funding allocation to the regions 
and states. In the past this date has come ip February, although 
with FY 1977 beginning October 1, instead of July 1, the date 
may be delayed. The results of the developitient of output 
standards for use in the funding allocation process must be 
available for use in the performance-based allocations. The 
th^rd milestone is June 3Cf, when the results of^ t^e input 
standards development should be available fox publication and 
distribution to the states. 

The following specific tasks and sche(^le are recommended ^s 
the minimum to implement static output and input standards. 

1. Develop Static, Output Performance Standards and 
\Proj action Model 

^. Data collection <9/l -,9/30) 

^ The basic d^ta to be used are Available for each state 

and for ISOInfajor labor areas. In actual practice 
output standards could be esrtablished for each of the 
150 labor areas and for 48 balance^-of-^state areas. 
" (Puerto Ric9 lacks data on External economic factors: 

Alaska performance is abnormal due to pipeline activity: 
Washington, D.C. is, 4 metropolitan area only; and 
Rhode Island consists almost entirely of the Providence 
SMSA . ) 
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(1) ES performance data is available from ESARS 
► Tables 12, 22 and 91B. At present, these 

tables are transmitted to the national office 
{0AM) fpr 80 SMSA's and 52 state agencies. ^It 
is reconmiended that the tables for FY 1975 be 
obtained for the remainder of the 150 major labor 
\ areas and ^or individual cost centers within them* 

(2) Eh cost data is available from the Cost Accounting 
System, but only state totals are currently sent 
to the national office. Preparation of man^year 
estimates by labor area requires that Report 03 

be obtained from each cost center using Title III 
grant funds in each labor area* 

The sum of the actual hours charged to 5xx and 
€xx time codes (exclusive of special projects 
within those codes) is the total hours applied- * 
to the placement process in each cost center* 

The SMSA total is the sum of the cost' centers 
within the SM^A* This total can be divided by 
any arbitrary standard work year ^such as 2040 
' hours) to obtain comparable estimates of man-years 
worked, adjusted for differential leave rates and 
hours of W(^k* 

(3) Annual labor force data (employment and unemploy* 
i^nt) are published in the annual Manpower Report 
o^ the President i Monthly data are available from 
"Area Trends in Employment and Unem|)loyment. " 

(4) Labor force con^osition data is available only 
from tbe 1970 Census, and is published in 
convenient form in the Coutity and City Data 
Book, -19 7 2* 

(5) Indiastry composition, employment and growth data 
are published monthly in "Employment and Earnings" 
with annual averages in the Mav issue* ^ 

b. Data analysis and model definition (9/1 - 10/31) 

Xl) Develop structural e<5uations from the hypotheses 
in Figure li-*3* 
I ■ 

6 (2) Test hypotheses and structural relationships to 
identify the statistically significant environ-* 
mental factors and the best specification 
of" factor. 
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Test alternate methods for setting standards : 



(a) Develop equation(s) for computing 
adjusted performance averages, 

(b) Develop typology and compute averages 
and standard deviations within type* 

(4) Combine data from cost* centers to level of SMSA 
parts and test methods for setting s;tandards in 
SMSA parts of* multi-state areas t 

Implement output standards model ill/1 - 12/31) 

(1) Write computer programs to implement, the 
accepted /nethod for letting output standards, 
tfie methods for compearing actual performance 
with the stantSerd, and the method for computing 
state summary spores, 

(2) Compute expected values and deviations between 
actual and expected performance for each labor 
area afnd "state fgr FY 19 75* Publish standards 
for dissemination/ 

(3) Write programs to cotjipute resource allocations, 
total and marginal, 1^ state* 

o 

(4) Design an es output simulation model ^nd simulate 
ES perforrnange under different resources 
allocation assfimptions. 

Develop ES Input Standards (9/1/75 - 8/31/76) 

(1) Select a stratified sample of - areas using ii^itial 
dutput standards and typology fco identify high . 
performing areas by type of labor market, jphe^ 
recommended sample would include about 31 areas, 
200 offices and 2 ,000 staff (12/1/75 12/31/75) 

(2) Prepare regions and statg^ to collect data ] 
i(2/l/75 - 2/28/76) 

(a) Publish list of sample a^eas* 

(b) Consult with regions and states. 

(c) Train -regional survey rteams • 

(3) Data collection (3/1/76 - 3/30/76) 

(a) Collect data on time utilization and quality 
factors using Handbook techniques and 
materials from Volume 4 of this report, 
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(b) Collect performance data^ from sample 
/ LOs for use in computing service ratios* 

Data analysis (4/1/76 - 5/31/76) 

(a) Analyze time distributions vs* cost , 
.accounting system* 

(b) Analyze time dist;ributions , service r;^tios,^ 
quality factors, etc* . 

Establish input performance standkrds by type of 
labor area, publish standards, and publish list 
of labor areas by type of area (6/1/75^- 6/30/7^) 

e* implement Performance Standards Utilization System 
{10/1/75 8/30/76) 

(1) Provide handbooks and train regional staff to 
interpret output staridards and apply them to 

f labor areas * ^ 

^ (2) Provide handbooks and guidelines for applying 
input standards and other diaignOstic t6ols* 

(3) Provide training" and technical assistance in 

developing performance improvement plans using 
an integrative apptoach.* 

Develop Dynamic Output Performance Standards and 
Projection Model 

In addition, it is recommended that work begin in parallel 
On the development of a dynamic output performance 
standards and projection model, as a crucial next ?tep in 
developing toqls for evaluation and (possibly) fundin® 
allocation that adequately reflect the effects of inter- 
temporal variations in economic conditions on ES 
performance* Some of the tasks in the development might' 
be as follows : 

a* Model definition 

M) Define hypotheses for time-dependent Relationships 
|>etween extremal factors and ES performance: 
stable economic factors, volatile economic factors 
and policy factors. 

(2) Specify structural equations in an ideal model* 

(3) Specify equations in a practical model* 

b* Data collection may be limited to tecent periods 

during which national ES policy was relatively constant 
and for which performance data are available* Ufc 
may be feasible fco collect data from the sources^ 
listed above for the following: 

"■SI 
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(1) Quarterly performance data from 150 major labor 
areas ^nd from balance-of-state areas for 
7/1/74 to 12/31/75- ^ ^ 

(2) Quarterly data oa external factors for the same 
areas from 1/1/74 to 12/31/75. 

(3) Quarterly staffing data for 7/1/74 to 12/31/75. 

c. ^ Data analysis c 

(y ^alyze quarterly changes and rates of change 

"^of external factors in relatioiji to ES per^formance , 

(2) Test alternate lag factors. 

(3) Test equations in the structural model. 

(4) Devfelop seasonal adjustments. 

■ d. Finalize dynamic model for use 

^ -(1) Simulate expected performance and deviations for 

FY 75. 

(2) Project performance for the ist half of FY 76 by 
'State and labor area, and compare actual with 

^ projected. 

(3) Compute optimum iresource allocations. 

(4) Simulate ES response to changes in resource 

. allocations, economic conditions and policies. 

One of the most important tasks In the development of ES ^ 
performance standards is not listed in the above schediile. 
That is the task^pf maintainijig close contact with ES manage- 
ment at the national, regional and state level. Unlike many 
other research and development efforts, the development of 
ES performance standards is decision^driven: the results, will 
be us^d' to make decisions about ES budgets and resource 
allocations almost as quickly as the products are available. 
The development program will utilize methods that are ^ 
unfamiliar, to rrtiany^.and will yield results that can have a 
revolutionary/ impact on the ES system. For this reason, it 
is recommended* that specific resources be earmarked for the 
purpose of supporting immediate uses of result^ in the BPF 
and other applications in parallel .with developipent tasks. 

Certair) significant ' policy (^est-ions ha,ve already been 
identified and others will aarise that Will require decisions 
to permit the program to proceed. Furthermore, the ptdducts 
of the program should be delivered ^ ^tate and local 
administrators in a form that is /un^rst^ and usable 
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i;ithout extensive on-site technical assistance. ES officials 
at all levels have an excellent practical instinct for 
identifying research and developme'ht results thaft do not 
'*make serfse.'* 

Close consultation with^ES decision-makers at all levels 
can help to insure that the development ]f)rograin riemains 
grounded in, reality and that usable products can be developed 
within the very tight schedule. It is recommended that a 
full time project monitor be assigne*? for the prograun, to meet 
at least "weekly with the project director. Monthly briefings 
should be held with all Associate Manpower Administrators or 
their assistants to review work completed during the past 
jnonth and the schedule for the coming month. trS addition f it 
is recommended that a steeripg committee including national, 
^regional and state" ES officials be established to meet and - 
review signj^ficant .products . These st$p9 . can help to insure 
timely .delivery of usable results on a schedule that meets ^ 
the needs bf decision^^makers. 

Further DeXrelopment Steps . . 

At the end of the ohe-ygar program outlined above, usable 

performance stan da rds wi]J.,iiave^,been. developed,- -bu%-^ ther^ 

will remain the need for further research and refinement. 

^Although 7\o specific schedule can be developed at this time, 

'it is us^eful to suinmarize the major items. 

1. Validation and Revalidation 6f Performance Standards 

The need for idatlon and xevalid^ation of performance 
standards relates to both outpy^: and input, standards. 
For output standards, validation implies further r^earch 
and examinatio/i of the basic model for projecting ES 
output performance to test the relationships - between 
factors hypothesized'tb have an effect ori performanc'e 
*and actual performance. ^ As data become available on 
additional factors, thelse will need to be tested and 
incorporated into the m|odel. In addition, as more time 
series data become available on ES performance, the ^ 
parameters of the dynamic model will need re-estimation. 
Finally as-MTesearch proceeds on the specification of 
ES objectives and measuring^ performance in relation to' 
those objectives, the basic .elements of the model may 
change . . \ i _ ^ _ 

For input standards, it \i^s already been stated that it 
is not po^'sible to validate the standards at the present 
time, because the optirri^ configuration of input resources 
depends Aipon other, prior in^iernal factors such as the mix 
of applicants and openings received, the location of -i 
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ES offices, ahd 'the overall qUality of ES management. 
Validatiol^^Jm the future will require that " methods 'be 
used, to control for such factors and that data on input 
utilization be collected from a" sample of offices 
representative of the full range of output performance, 
external factors dnd ifiternal factors. Revalidation of 
input standards, once established, will be needed to 
ascertain whether or not changes in policies and manage- 
ment approaches have altered the patterns of input 
utilization that discriminate amang high and low performing 
areas* Revalidation will require new purveys. These 
could be done on a three-year rotation basis, revalidating 
selected types of areas. each year. It is recommended 
that a revalidation program be integrated with a quarterly 
ES staff time utilization sample,^ 

2. Integration with other research and development results 

Research is* currently underway to test methods for 
estimating ES performance potential* During the coming 
year, there will be a;t least five research and evaluation 
projects undertaken to improve measures of tS productivity; 
to examine the effects of mandatory registrations and 
otTier ES enforcement "activities on ES pafOductivity; and 
to evaluate counselling, job information services and 
employer services* During the past year, methods have 
been developed Or refined for assisting in the location 
of local offices and for evaluating significant dimensions 
of the overall quality of management* While specific 
steps can and should be taJcen to coordinate the develop* 
ment of performance standards with all of these, it is 
virtually certain ^that findings will emerge during the 
next year that c^n only be integrated irito the overall" 
ES management system As a ^art of a continuing program 
for refinement and improvement* 



G* Limitations of Performance Standards , 

- f 
The basic purpose of performance standards is to provide 
guides and methods for improving .the. quality and quantity 
of service provided by the Employment Service. These ax^ 
other approaches ,which aim at the improvement of Employment 
Service" performance must take into account realistic limits 
on the improvements achievable by ES manageinent* ^ Many ^ 
factors affect the operations of a local ES such as the 
external lat^or market, the general ecC>nomic conditions in 
the community, the attitudes of the staff, and the skill 
levels of the- staff. Th^se internal and external factors 
combine to create a .style of operati*bns and a pattern of 
achievement for each local office. Past history, past 
perceptions, and past pQlicie^ also effect this picture - 
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even when present conditions have been radically changed ^ 
and even wh^n considerable time has elapsed 9 since past 
conditions existed* Much of the actual operations are 
determined by^ factors -external to the ES or out of the 
control of local managers; therefore current management 
practices can influence only a fraction of total achieve- 
ment* 

The input stjandards focus upon the internal operations- of 
the ES in a local office or labor area; they>£an only be 
used to improve that -fraction of total achi^ement influenced 
by local internal factors* The methodology treats these 
internal factors in an analytical or reductive manner* It 
breaks, down the placement process into component activities 
and treats each as a separate component* The result of 
applying the method yields ^ a measure of the separate component 
activities actually carried on in the local Employment Service 
and a comparison of those * activities with "standards"* 

Differences between the actual and .standard" activities are 
then interpreted as Clues for spotting weak areas and for 
developing a plan for improvement* - Operations in th^ loc^^ 
office , however , actually invqlve a good deal p.f interaction 
among these componenfee and with all -the external and iirtema^l 
conditions influencing ES performance* Effects of the inter^ 
action among components are not directly measured by.thp' 
recommended methodology* Therefore," performance standards 
deal only with a portion of these many factors subject tb . * , 
objective measurement and numeric expression** (The quaj^it^'ive 
^EOeasurements that have been developed are tied to separ|±e^ 
component activities and do not directly effect tl;ie \o;/e^cfii,l' 
environment in which the ES functions*) *^ 

The potential for improvement , however , must , be expressed^ " 
and accounted for in any improvem^t plan, in answer to this, 
question: What can we expect this local Employment Service 
to achieve, given the external faQtors in the community? 
One currently feasible way to answer this question is to. . 
apply statistical analysis to develop performance adjustment 
equations; another is tj group areas , based 6a individual 
external factors* Bo^i methods yield output standards that 
are derived from a best fit to the data on the effects of 
external variable^ and on measures of ES' performance* The 
results tell xj^what an average office operating in a particular 
environment would achieve, given the external factors* The 
differences between expected and actual achievement should be 
entirely due to differences in internal ES factors between the 
particular local officfe and the average* * 



However, neither method' for setting output standards is based 
on a complete model ^of causality. Even after applying the 
methods, we might still ask how these externai'^f actors 
operate on the expectations and work patterns of the local 
office staff. A strictly statistical approach may not provide 
^insights into these human factors. We may find that the 
patterns revealed by measuring septate activities lead to 
questions that can be answered only after investigating tlie ^ 
' human integrative issues in the Employment Service. Such 
integrative issues are particularly significant for the 
manager - the key individual in the local office. State 
administrators have often said th'^t if they knew the name of 
the" manager they could tell how the local office has performed. 
The analytical, reductive approach might reflect these 
holl^tijc factors indirectly but, by itself, it may not provide 
the key information fot an improvement pl^n. Therefore we 
should apply the performance standards methods cautiously 
while maintaining a sensitivity for the broader effects. The 
asic purposes of the Employment Service are important in 
AgT^his context. We might ask: What is this office trying to 
Jw/Ao7 What are the goals of the ES in this community as 
j^r^i perceived by, the local managers? How ate they measuring the 
'^^"^/achievement of the E^S^^in meeting these goals'and are these 
the appropriate goa^i^ ^.^ 

The ES output performance goals uhderiying the recommended 
methods for establishing performance standards are as expressed 
r in the BalancecP^piacement Formula. If the ptate or the local 

managers of the Employment Service are not trying to achieve 
the goalP represented by this formula the diagnostic methods 
of the input standards may not apply. If they a re trying to 
meet these goals and if they accept the BPF-type measures as 
y appropriate and performance standards measures of component 

activities as sound, we believe the method will lead to 
improvements. ^ ^ ^ ' 

But the iTiput standards must be applied cautiously, with these 
cavea^ts in mind, and with an understanding of. the holistic 
relationships; otherwise, a mechanical application of the 
method will be unrewarding. For example problems might derive 
from the morale and skill level of the local staff which are ^ 
cause^ by such factors as the prevailing salaried, S'tate Civil ^ 
Service policies, and past practices ir\ the State agency. The 
performance Standards cannot directly measure these effects? 
it can only^eflect them- In other cases the achievement of 
>iigh ^ores on a BPF-type of measurement mi<3ht conf lict^ with 
programmatic goals of the a^^ncy. tn our survey of 18 local 
offices we observed such practices, but we have not beei^ able^ 
to quantify or codify these observations; they remain at the 
level of looal situational observations. The foregoing 





discussion of the holistic effects of ES organizational 
behavior lead us to expect that such practices cannot be 
made explicit through a reductive method such as performance 
'Standards. If these holistifi effects are over-riding the 
effects of the measurable par^s as detenainants of ,ES 
achievement, if they underly everything else, then we have 
only toegun the research that is needed: research on the 

integratj.ve effects. 

^ - 

The problems in applying performance standards are parallel 
to problems in sociological search associated with the 
controversy over "operationalism" that reached its height in 
the 1930's and 1940's. In summary this controversy centered 
around the differences between measurable quantities and - 
theoretical concepts . * Our problem is similar because we are 
trying to express ;the underlying goal and the causal Relation- 
ships between E5 staff organizational behavior and achievement 
of those goals by reducing these factors (goals- and organiza- 
tional behav4iOr) to measurable quantities which are then the 
data of our method. The, capacity to truly develop insights 
into ES achievement through the use of these methods depends 
upon the validity of such measurements in expressing the 
achievement of goals and the behavior of the ES. Our approach 
(largely analytical and reductive) necessarily eliminates 
integrative factors, indeed: even if we could measure those 
factors we anticipate that th6 techniques would be too com- 
^plicated for practical application* ' *' 

The Performance Standards Method is offered here as a tool in 
the armamentarium of the ES manager; it is no t the total 
answer to management's needs for analysis* In some cases the 
problems are caused by " political" and socio-economic factors 
that underly the measurement of component activity. Effective 
planning fpr , improvement should recognize these^ cases and the 
connections between the "political" factors ana component 
activities ; the efforts for change should then be applied to 
those factors that u^d^rly the problem. 
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*FQr a detailed discussion of these problems see HtM* Blalock and 
A.B. Blalock "Methodology in Social Research". Meb^^aw-HiLl, 1968. 



